^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) Written by: Neil Brown
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) Please see MAINTAINERS file for where to send questions.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) Overlay Filesystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) ==================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) This document describes a prototype for a new approach to providing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) overlay-filesystem functionality in Linux (sometimes referred to as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) union-filesystems). An overlay-filesystem tries to present a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) filesystem which is the result over overlaying one filesystem on top
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) of the other.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) Overlay objects
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) The overlay filesystem approach is 'hybrid', because the objects that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) appear in the filesystem do not always appear to belong to that filesystem.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) In many cases, an object accessed in the union will be indistinguishable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) from accessing the corresponding object from the original filesystem.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) This is most obvious from the 'st_dev' field returned by stat(2).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) While directories will report an st_dev from the overlay-filesystem,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) non-directory objects may report an st_dev from the lower filesystem or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) upper filesystem that is providing the object. Similarly st_ino will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) only be unique when combined with st_dev, and both of these can change
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) over the lifetime of a non-directory object. Many applications and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) tools ignore these values and will not be affected.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) In the special case of all overlay layers on the same underlying
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) filesystem, all objects will report an st_dev from the overlay
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) filesystem and st_ino from the underlying filesystem. This will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) make the overlay mount more compliant with filesystem scanners and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) overlay objects will be distinguishable from the corresponding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) objects in the original filesystem.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) On 64bit systems, even if all overlay layers are not on the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) underlying filesystem, the same compliant behavior could be achieved
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) with the "xino" feature. The "xino" feature composes a unique object
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) identifier from the real object st_ino and an underlying fsid index.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) If all underlying filesystems support NFS file handles and export file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) handles with 32bit inode number encoding (e.g. ext4), overlay filesystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46) will use the high inode number bits for fsid. Even when the underlying
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) filesystem uses 64bit inode numbers, users can still enable the "xino"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) feature with the "-o xino=on" overlay mount option. That is useful for the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) case of underlying filesystems like xfs and tmpfs, which use 64bit inode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) numbers, but are very unlikely to use the high inode number bits. In case
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) the underlying inode number does overflow into the high xino bits, overlay
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) filesystem will fall back to the non xino behavior for that inode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) The following table summarizes what can be expected in different overlay
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) configurations.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) Inode properties
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) ````````````````
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) +--------------+------------+------------+-----------------+----------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) |Configuration | Persistent | Uniform | st_ino == d_ino | d_ino == i_ino |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) | | st_ino | st_dev | | [*] |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) +==============+=====+======+=====+======+========+========+========+=======+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) | | dir | !dir | dir | !dir | dir + !dir | dir | !dir |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) +--------------+-----+------+-----+------+--------+--------+--------+-------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) | All layers | Y | Y | Y | Y | Y | Y | Y | Y |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) | on same fs | | | | | | | | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) +--------------+-----+------+-----+------+--------+--------+--------+-------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) | Layers not | N | Y | Y | N | N | Y | N | Y |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) | on same fs, | | | | | | | | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) | xino=off | | | | | | | | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) +--------------+-----+------+-----+------+--------+--------+--------+-------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) | xino=on/auto | Y | Y | Y | Y | Y | Y | Y | Y |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) | | | | | | | | | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) +--------------+-----+------+-----+------+--------+--------+--------+-------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) | xino=on/auto,| N | Y | Y | N | N | Y | N | Y |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) | ino overflow | | | | | | | | |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) +--------------+-----+------+-----+------+--------+--------+--------+-------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) [*] nfsd v3 readdirplus verifies d_ino == i_ino. i_ino is exposed via several
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) /proc files, such as /proc/locks and /proc/self/fdinfo/<fd> of an inotify
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) file descriptor.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) Upper and Lower
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) An overlay filesystem combines two filesystems - an 'upper' filesystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) and a 'lower' filesystem. When a name exists in both filesystems, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) object in the 'upper' filesystem is visible while the object in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) 'lower' filesystem is either hidden or, in the case of directories,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) merged with the 'upper' object.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) It would be more correct to refer to an upper and lower 'directory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) tree' rather than 'filesystem' as it is quite possible for both
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) directory trees to be in the same filesystem and there is no
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) requirement that the root of a filesystem be given for either upper or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) lower.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) The lower filesystem can be any filesystem supported by Linux and does
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) not need to be writable. The lower filesystem can even be another
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) overlayfs. The upper filesystem will normally be writable and if it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) is it must support the creation of trusted.* extended attributes, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) must provide valid d_type in readdir responses, so NFS is not suitable.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) A read-only overlay of two read-only filesystems may use any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) filesystem type.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) Directories
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) -----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) Overlaying mainly involves directories. If a given name appears in both
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) upper and lower filesystems and refers to a non-directory in either,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) then the lower object is hidden - the name refers only to the upper
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) object.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) Where both upper and lower objects are directories, a merged directory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) is formed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) At mount time, the two directories given as mount options "lowerdir" and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) "upperdir" are combined into a merged directory:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) mount -t overlay overlay -olowerdir=/lower,upperdir=/upper,\
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) workdir=/work /merged
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) The "workdir" needs to be an empty directory on the same filesystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) as upperdir.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) Then whenever a lookup is requested in such a merged directory, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) lookup is performed in each actual directory and the combined result
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) is cached in the dentry belonging to the overlay filesystem. If both
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) actual lookups find directories, both are stored and a merged
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) directory is created, otherwise only one is stored: the upper if it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) exists, else the lower.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) Only the lists of names from directories are merged. Other content
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) such as metadata and extended attributes are reported for the upper
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) directory only. These attributes of the lower directory are hidden.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) credentials
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) -----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) By default, all access to the upper, lower and work directories is the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) recorded mounter's MAC and DAC credentials. The incoming accesses are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) checked against the caller's credentials.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) In the case where caller MAC or DAC credentials do not overlap, a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) use case available in older versions of the driver, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) override_creds mount flag can be turned off and help when the use
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) pattern has caller with legitimate credentials where the mounter
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) does not. Several unintended side effects will occur though. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) caller without certain key capabilities or lower privilege will not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) always be able to delete files or directories, create nodes, or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) search some restricted directories. The ability to search and read
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) a directory entry is spotty as a result of the cache mechanism not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) retesting the credentials because of the assumption, a privileged
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) caller can fill cache, then a lower privilege can read the directory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) cache. The uneven security model where cache, upperdir and workdir
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) are opened at privilege, but accessed without creating a form of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) privilege escalation, should only be used with strict understanding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) of the side effects and of the security policies.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) whiteouts and opaque directories
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) --------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) In order to support rm and rmdir without changing the lower
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) filesystem, an overlay filesystem needs to record in the upper filesystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) that files have been removed. This is done using whiteouts and opaque
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169) directories (non-directories are always opaque).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) A whiteout is created as a character device with 0/0 device number.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) When a whiteout is found in the upper level of a merged directory, any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) matching name in the lower level is ignored, and the whiteout itself
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) is also hidden.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) A directory is made opaque by setting the xattr "trusted.overlay.opaque"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) to "y". Where the upper filesystem contains an opaque directory, any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) directory in the lower filesystem with the same name is ignored.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) readdir
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) -------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) When a 'readdir' request is made on a merged directory, the upper and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) lower directories are each read and the name lists merged in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) obvious way (upper is read first, then lower - entries that already
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) exist are not re-added). This merged name list is cached in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) 'struct file' and so remains as long as the file is kept open. If the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) directory is opened and read by two processes at the same time, they
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) will each have separate caches. A seekdir to the start of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) directory (offset 0) followed by a readdir will cause the cache to be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) discarded and rebuilt.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) This means that changes to the merged directory do not appear while a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) directory is being read. This is unlikely to be noticed by many
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) programs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) seek offsets are assigned sequentially when the directories are read.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) Thus if
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) - read part of a directory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) - remember an offset, and close the directory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) - re-open the directory some time later
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) - seek to the remembered offset
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) there may be little correlation between the old and new locations in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) the list of filenames, particularly if anything has changed in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) directory.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) Readdir on directories that are not merged is simply handled by the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210) underlying directory (upper or lower).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) renaming directories
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) --------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) When renaming a directory that is on the lower layer or merged (i.e. the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) directory was not created on the upper layer to start with) overlayfs can
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) handle it in two different ways:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) 1. return EXDEV error: this error is returned by rename(2) when trying to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) move a file or directory across filesystem boundaries. Hence
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) applications are usually prepared to hande this error (mv(1) for example
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222) recursively copies the directory tree). This is the default behavior.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224) 2. If the "redirect_dir" feature is enabled, then the directory will be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) copied up (but not the contents). Then the "trusted.overlay.redirect"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) extended attribute is set to the path of the original location from the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227) root of the overlay. Finally the directory is moved to the new
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) location.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230) There are several ways to tune the "redirect_dir" feature.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) Kernel config options:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234) - OVERLAY_FS_REDIRECT_DIR:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235) If this is enabled, then redirect_dir is turned on by default.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) - OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) If this is enabled, then redirects are always followed by default. Enabling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) this results in a less secure configuration. Enable this option only when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) worried about backward compatibility with kernels that have the redirect_dir
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240) feature and follow redirects even if turned off.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) Module options (can also be changed through /sys/module/overlay/parameters/):
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) - "redirect_dir=BOOL":
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) See OVERLAY_FS_REDIRECT_DIR kernel config option above.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) - "redirect_always_follow=BOOL":
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247) See OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW kernel config option above.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) - "redirect_max=NUM":
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249) The maximum number of bytes in an absolute redirect (default is 256).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251) Mount options:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) - "redirect_dir=on":
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254) Redirects are enabled.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255) - "redirect_dir=follow":
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) Redirects are not created, but followed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) - "redirect_dir=off":
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) Redirects are not created and only followed if "redirect_always_follow"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) feature is enabled in the kernel/module config.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260) - "redirect_dir=nofollow":
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261) Redirects are not created and not followed (equivalent to "redirect_dir=off"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262) if "redirect_always_follow" feature is not enabled).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264) When the NFS export feature is enabled, every copied up directory is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265) indexed by the file handle of the lower inode and a file handle of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266) upper directory is stored in a "trusted.overlay.upper" extended attribute
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267) on the index entry. On lookup of a merged directory, if the upper
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268) directory does not match the file handle stores in the index, that is an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269) indication that multiple upper directories may be redirected to the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270) lower directory. In that case, lookup returns an error and warns about
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271) a possible inconsistency.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273) Because lower layer redirects cannot be verified with the index, enabling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274) NFS export support on an overlay filesystem with no upper layer requires
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275) turning off redirect follow (e.g. "redirect_dir=nofollow").
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278) Non-directories
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279) ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281) Objects that are not directories (files, symlinks, device-special
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282) files etc.) are presented either from the upper or lower filesystem as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283) appropriate. When a file in the lower filesystem is accessed in a way
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284) the requires write-access, such as opening for write access, changing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285) some metadata etc., the file is first copied from the lower filesystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286) to the upper filesystem (copy_up). Note that creating a hard-link
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287) also requires copy_up, though of course creation of a symlink does
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 288) not.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 289)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 290) The copy_up may turn out to be unnecessary, for example if the file is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 291) opened for read-write but the data is not modified.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 292)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 293) The copy_up process first makes sure that the containing directory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 294) exists in the upper filesystem - creating it and any parents as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 295) necessary. It then creates the object with the same metadata (owner,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 296) mode, mtime, symlink-target etc.) and then if the object is a file, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 297) data is copied from the lower to the upper filesystem. Finally any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 298) extended attributes are copied up.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 299)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 300) Once the copy_up is complete, the overlay filesystem simply
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 301) provides direct access to the newly created file in the upper
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 302) filesystem - future operations on the file are barely noticed by the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 303) overlay filesystem (though an operation on the name of the file such as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 304) rename or unlink will of course be noticed and handled).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 305)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 306)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 307) Permission model
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 308) ----------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 309)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 310) Permission checking in the overlay filesystem follows these principles:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 311)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 312) 1) permission check SHOULD return the same result before and after copy up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 313)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 314) 2) task creating the overlay mount MUST NOT gain additional privileges
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 315)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 316) 3) non-mounting task MAY gain additional privileges through the overlay,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 317) compared to direct access on underlying lower or upper filesystems
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 318)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 319) This is achieved by performing two permission checks on each access
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 320)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 321) a) check if current task is allowed access based on local DAC (owner,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 322) group, mode and posix acl), as well as MAC checks
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 323)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 324) b) check if mounting task would be allowed real operation on lower or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 325) upper layer based on underlying filesystem permissions, again including
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 326) MAC checks
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 327)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 328) Check (a) ensures consistency (1) since owner, group, mode and posix acls
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 329) are copied up. On the other hand it can result in server enforced
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 330) permissions (used by NFS, for example) being ignored (3).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 331)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 332) Check (b) ensures that no task gains permissions to underlying layers that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 333) the mounting task does not have (2). This also means that it is possible
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 334) to create setups where the consistency rule (1) does not hold; normally,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 335) however, the mounting task will have sufficient privileges to perform all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 336) operations.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 337)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 338) Another way to demonstrate this model is drawing parallels between
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 339)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 340) mount -t overlay overlay -olowerdir=/lower,upperdir=/upper,... /merged
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 341)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 342) and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 343)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 344) cp -a /lower /upper
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 345) mount --bind /upper /merged
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 346)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 347) The resulting access permissions should be the same. The difference is in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 348) the time of copy (on-demand vs. up-front).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 349)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 350)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 351) Multiple lower layers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 352) ---------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 353)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 354) Multiple lower layers can now be given using the colon (":") as a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 355) separator character between the directory names. For example:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 356)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 357) mount -t overlay overlay -olowerdir=/lower1:/lower2:/lower3 /merged
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 358)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 359) As the example shows, "upperdir=" and "workdir=" may be omitted. In
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 360) that case the overlay will be read-only.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 361)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 362) The specified lower directories will be stacked beginning from the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 363) rightmost one and going left. In the above example lower1 will be the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 364) top, lower2 the middle and lower3 the bottom layer.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 365)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 366)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 367) Metadata only copy up
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 368) ---------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 369)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 370) When metadata only copy up feature is enabled, overlayfs will only copy
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 371) up metadata (as opposed to whole file), when a metadata specific operation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 372) like chown/chmod is performed. Full file will be copied up later when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 373) file is opened for WRITE operation.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 374)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 375) In other words, this is delayed data copy up operation and data is copied
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 376) up when there is a need to actually modify data.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 377)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 378) There are multiple ways to enable/disable this feature. A config option
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 379) CONFIG_OVERLAY_FS_METACOPY can be set/unset to enable/disable this feature
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 380) by default. Or one can enable/disable it at module load time with module
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 381) parameter metacopy=on/off. Lastly, there is also a per mount option
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 382) metacopy=on/off to enable/disable this feature per mount.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 383)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 384) Do not use metacopy=on with untrusted upper/lower directories. Otherwise
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 385) it is possible that an attacker can create a handcrafted file with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 386) appropriate REDIRECT and METACOPY xattrs, and gain access to file on lower
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 387) pointed by REDIRECT. This should not be possible on local system as setting
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 388) "trusted." xattrs will require CAP_SYS_ADMIN. But it should be possible
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 389) for untrusted layers like from a pen drive.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 390)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 391) Note: redirect_dir={off|nofollow|follow[*]} and nfs_export=on mount options
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 392) conflict with metacopy=on, and will result in an error.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 393)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 394) [*] redirect_dir=follow only conflicts with metacopy=on if upperdir=... is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 395) given.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 396)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 397) Sharing and copying layers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 398) --------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 399)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 400) Lower layers may be shared among several overlay mounts and that is indeed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 401) a very common practice. An overlay mount may use the same lower layer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 402) path as another overlay mount and it may use a lower layer path that is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 403) beneath or above the path of another overlay lower layer path.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 404)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 405) Using an upper layer path and/or a workdir path that are already used by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 406) another overlay mount is not allowed and may fail with EBUSY. Using
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 407) partially overlapping paths is not allowed and may fail with EBUSY.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 408) If files are accessed from two overlayfs mounts which share or overlap the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 409) upper layer and/or workdir path the behavior of the overlay is undefined,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 410) though it will not result in a crash or deadlock.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 411)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 412) Mounting an overlay using an upper layer path, where the upper layer path
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 413) was previously used by another mounted overlay in combination with a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 414) different lower layer path, is allowed, unless the "inodes index" feature
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 415) or "metadata only copy up" feature is enabled.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 416)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 417) With the "inodes index" feature, on the first time mount, an NFS file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 418) handle of the lower layer root directory, along with the UUID of the lower
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 419) filesystem, are encoded and stored in the "trusted.overlay.origin" extended
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 420) attribute on the upper layer root directory. On subsequent mount attempts,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 421) the lower root directory file handle and lower filesystem UUID are compared
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 422) to the stored origin in upper root directory. On failure to verify the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 423) lower root origin, mount will fail with ESTALE. An overlayfs mount with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 424) "inodes index" enabled will fail with EOPNOTSUPP if the lower filesystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 425) does not support NFS export, lower filesystem does not have a valid UUID or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 426) if the upper filesystem does not support extended attributes.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 427)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 428) For "metadata only copy up" feature there is no verification mechanism at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 429) mount time. So if same upper is mounted with different set of lower, mount
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 430) probably will succeed but expect the unexpected later on. So don't do it.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 431)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 432) It is quite a common practice to copy overlay layers to a different
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 433) directory tree on the same or different underlying filesystem, and even
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 434) to a different machine. With the "inodes index" feature, trying to mount
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 435) the copied layers will fail the verification of the lower root file handle.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 436)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 437)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 438) Non-standard behavior
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 439) ---------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 440)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 441) Current version of overlayfs can act as a mostly POSIX compliant
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 442) filesystem.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 443)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 444) This is the list of cases that overlayfs doesn't currently handle:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 445)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 446) a) POSIX mandates updating st_atime for reads. This is currently not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 447) done in the case when the file resides on a lower layer.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 448)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 449) b) If a file residing on a lower layer is opened for read-only and then
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 450) memory mapped with MAP_SHARED, then subsequent changes to the file are not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 451) reflected in the memory mapping.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 452)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 453) The following options allow overlayfs to act more like a standards
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 454) compliant filesystem:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 455)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 456) 1) "redirect_dir"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 457)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 458) Enabled with the mount option or module option: "redirect_dir=on" or with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 459) the kernel config option CONFIG_OVERLAY_FS_REDIRECT_DIR=y.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 460)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 461) If this feature is disabled, then rename(2) on a lower or merged directory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 462) will fail with EXDEV ("Invalid cross-device link").
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 463)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 464) 2) "inode index"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 465)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 466) Enabled with the mount option or module option "index=on" or with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 467) kernel config option CONFIG_OVERLAY_FS_INDEX=y.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 468)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 469) If this feature is disabled and a file with multiple hard links is copied
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 470) up, then this will "break" the link. Changes will not be propagated to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 471) other names referring to the same inode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 472)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 473) 3) "xino"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 474)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 475) Enabled with the mount option "xino=auto" or "xino=on", with the module
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 476) option "xino_auto=on" or with the kernel config option
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 477) CONFIG_OVERLAY_FS_XINO_AUTO=y. Also implicitly enabled by using the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 478) underlying filesystem for all layers making up the overlay.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 479)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 480) If this feature is disabled or the underlying filesystem doesn't have
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 481) enough free bits in the inode number, then overlayfs will not be able to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 482) guarantee that the values of st_ino and st_dev returned by stat(2) and the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 483) value of d_ino returned by readdir(3) will act like on a normal filesystem.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 484) E.g. the value of st_dev may be different for two objects in the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 485) overlay filesystem and the value of st_ino for directory objects may not be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 486) persistent and could change even while the overlay filesystem is mounted, as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 487) summarized in the `Inode properties`_ table above.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 488)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 489)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 490) Changes to underlying filesystems
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 491) ---------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 492)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 493) Offline changes, when the overlay is not mounted, are allowed to either
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 494) the upper or the lower trees.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 495)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 496) Changes to the underlying filesystems while part of a mounted overlay
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 497) filesystem are not allowed. If the underlying filesystem is changed,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 498) the behavior of the overlay is undefined, though it will not result in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 499) a crash or deadlock.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 500)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 501) When the overlay NFS export feature is enabled, overlay filesystems
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 502) behavior on offline changes of the underlying lower layer is different
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 503) than the behavior when NFS export is disabled.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 504)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 505) On every copy_up, an NFS file handle of the lower inode, along with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 506) UUID of the lower filesystem, are encoded and stored in an extended
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 507) attribute "trusted.overlay.origin" on the upper inode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 508)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 509) When the NFS export feature is enabled, a lookup of a merged directory,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 510) that found a lower directory at the lookup path or at the path pointed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 511) to by the "trusted.overlay.redirect" extended attribute, will verify
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 512) that the found lower directory file handle and lower filesystem UUID
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 513) match the origin file handle that was stored at copy_up time. If a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 514) found lower directory does not match the stored origin, that directory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 515) will not be merged with the upper directory.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 516)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 517)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 518)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 519) NFS export
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 520) ----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 521)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 522) When the underlying filesystems supports NFS export and the "nfs_export"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 523) feature is enabled, an overlay filesystem may be exported to NFS.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 524)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 525) With the "nfs_export" feature, on copy_up of any lower object, an index
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 526) entry is created under the index directory. The index entry name is the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 527) hexadecimal representation of the copy up origin file handle. For a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 528) non-directory object, the index entry is a hard link to the upper inode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 529) For a directory object, the index entry has an extended attribute
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 530) "trusted.overlay.upper" with an encoded file handle of the upper
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 531) directory inode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 532)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 533) When encoding a file handle from an overlay filesystem object, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 534) following rules apply:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 535)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 536) 1. For a non-upper object, encode a lower file handle from lower inode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 537) 2. For an indexed object, encode a lower file handle from copy_up origin
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 538) 3. For a pure-upper object and for an existing non-indexed upper object,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 539) encode an upper file handle from upper inode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 540)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 541) The encoded overlay file handle includes:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 542) - Header including path type information (e.g. lower/upper)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 543) - UUID of the underlying filesystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 544) - Underlying filesystem encoding of underlying inode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 545)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 546) This encoding format is identical to the encoding format file handles that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 547) are stored in extended attribute "trusted.overlay.origin".
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 548)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 549) When decoding an overlay file handle, the following steps are followed:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 550)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 551) 1. Find underlying layer by UUID and path type information.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 552) 2. Decode the underlying filesystem file handle to underlying dentry.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 553) 3. For a lower file handle, lookup the handle in index directory by name.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 554) 4. If a whiteout is found in index, return ESTALE. This represents an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 555) overlay object that was deleted after its file handle was encoded.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 556) 5. For a non-directory, instantiate a disconnected overlay dentry from the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 557) decoded underlying dentry, the path type and index inode, if found.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 558) 6. For a directory, use the connected underlying decoded dentry, path type
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 559) and index, to lookup a connected overlay dentry.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 560)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 561) Decoding a non-directory file handle may return a disconnected dentry.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 562) copy_up of that disconnected dentry will create an upper index entry with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 563) no upper alias.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 564)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 565) When overlay filesystem has multiple lower layers, a middle layer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 566) directory may have a "redirect" to lower directory. Because middle layer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 567) "redirects" are not indexed, a lower file handle that was encoded from the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 568) "redirect" origin directory, cannot be used to find the middle or upper
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 569) layer directory. Similarly, a lower file handle that was encoded from a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 570) descendant of the "redirect" origin directory, cannot be used to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 571) reconstruct a connected overlay path. To mitigate the cases of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 572) directories that cannot be decoded from a lower file handle, these
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 573) directories are copied up on encode and encoded as an upper file handle.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 574) On an overlay filesystem with no upper layer this mitigation cannot be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 575) used NFS export in this setup requires turning off redirect follow (e.g.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 576) "redirect_dir=nofollow").
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 577)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 578) The overlay filesystem does not support non-directory connectable file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 579) handles, so exporting with the 'subtree_check' exportfs configuration will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 580) cause failures to lookup files over NFS.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 581)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 582) When the NFS export feature is enabled, all directory index entries are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 583) verified on mount time to check that upper file handles are not stale.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 584) This verification may cause significant overhead in some cases.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 585)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 586) Note: the mount options index=off,nfs_export=on are conflicting for a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 587) read-write mount and will result in an error.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 588)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 589)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 590) Volatile mount
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 591) --------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 592)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 593) This is enabled with the "volatile" mount option. Volatile mounts are not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 594) guaranteed to survive a crash. It is strongly recommended that volatile
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 595) mounts are only used if data written to the overlay can be recreated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 596) without significant effort.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 597)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 598) The advantage of mounting with the "volatile" option is that all forms of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 599) sync calls to the upper filesystem are omitted.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 600)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 601) In order to avoid a giving a false sense of safety, the syncfs (and fsync)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 602) semantics of volatile mounts are slightly different than that of the rest of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 603) VFS. If any writeback error occurs on the upperdir's filesystem after a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 604) volatile mount takes place, all sync functions will return an error. Once this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 605) condition is reached, the filesystem will not recover, and every subsequent sync
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 606) call will return an error, even if the upperdir has not experience a new error
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 607) since the last sync call.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 608)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 609) When overlay is mounted with "volatile" option, the directory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 610) "$workdir/work/incompat/volatile" is created. During next mount, overlay
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 611) checks for this directory and refuses to mount if present. This is a strong
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 612) indicator that user should throw away upper and work directories and create
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 613) fresh one. In very limited cases where the user knows that the system has
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 614) not crashed and contents of upperdir are intact, The "volatile" directory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 615) can be removed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 616)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 617) Testsuite
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 618) ---------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 619)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 620) There's a testsuite originally developed by David Howells and currently
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 621) maintained by Amir Goldstein at:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 622)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 623) https://github.com/amir73il/unionmount-testsuite.git
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 624)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 625) Run as root:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 626)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 627) # cd unionmount-testsuite
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 628) # ./run --ov --verify
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags