^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) ===================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) pNFS block layout server user guide
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) ===================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) The Linux NFS server now supports the pNFS block layout extension. In this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) case the NFS server acts as Metadata Server (MDS) for pNFS, which in addition
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) to handling all the metadata access to the NFS export also hands out layouts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) to the clients to directly access the underlying block devices that are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) shared with the client.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) To use pNFS block layouts with the Linux NFS server the exported file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) system needs to support the pNFS block layouts (currently just XFS), and the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) file system must sit on shared storage (typically iSCSI) that is accessible
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) to the clients in addition to the MDS. As of now the file system needs to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) sit directly on the exported volume, striping or concatenation of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) volumes on the MDS and clients is not supported yet.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) On the server, pNFS block volume support is automatically if the file system
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) support it. On the client make sure the kernel has the CONFIG_PNFS_BLOCK
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) option enabled, the blkmapd daemon from nfs-utils is running, and the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) file system is mounted using the NFSv4.1 protocol version (mount -o vers=4.1).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) If the nfsd server needs to fence a non-responding client it calls
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) /sbin/nfsd-recall-failed with the first argument set to the IP address of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) the client, and the second argument set to the device node without the /dev
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) prefix for the file system to be fenced. Below is an example file that shows
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) how to translate the device into a serial number from SCSI EVPD 0x80::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) cat > /sbin/nfsd-recall-failed << EOF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) .. code-block:: sh
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) #!/bin/sh
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) CLIENT="$1"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) DEV="/dev/$2"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) EVPD=`sg_inq --page=0x80 ${DEV} | \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) grep "Unit serial number:" | \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) awk -F ': ' '{print $2}'`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) echo "fencing client ${CLIENT} serial ${EVPD}" >> /var/log/pnfsd-fence.log
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) EOF