^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) .. include:: <isonum.txt>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) ====================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) PCI Express I/O Virtualization Howto
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) ====================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) :Copyright: |copy| 2009 Intel Corporation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) :Authors: - Yu Zhao <yu.zhao@intel.com>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) - Donald Dutile <ddutile@redhat.com>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) Overview
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) ========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) What is SR-IOV
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) --------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) Single Root I/O Virtualization (SR-IOV) is a PCI Express Extended
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) capability which makes one physical device appear as multiple virtual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) devices. The physical device is referred to as Physical Function (PF)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) while the virtual devices are referred to as Virtual Functions (VF).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) Allocation of the VF can be dynamically controlled by the PF via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) registers encapsulated in the capability. By default, this feature is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) not enabled and the PF behaves as traditional PCIe device. Once it's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) turned on, each VF's PCI configuration space can be accessed by its own
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) Bus, Device and Function Number (Routing ID). And each VF also has PCI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) Memory Space, which is used to map its register set. VF device driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) operates on the register set so it can be functional and appear as a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) real existing PCI device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) User Guide
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) ==========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) How can I enable SR-IOV capability
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) ----------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) Multiple methods are available for SR-IOV enablement.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) In the first method, the device driver (PF driver) will control the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) enabling and disabling of the capability via API provided by SR-IOV core.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) If the hardware has SR-IOV capability, loading its PF driver would
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) enable it and all VFs associated with the PF. Some PF drivers require
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) a module parameter to be set to determine the number of VFs to enable.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) In the second method, a write to the sysfs file sriov_numvfs will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) enable and disable the VFs associated with a PCIe PF. This method
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) enables per-PF, VF enable/disable values versus the first method,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46) which applies to all PFs of the same device. Additionally, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) PCI SRIOV core support ensures that enable/disable operations are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) valid to reduce duplication in multiple drivers for the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) checks, e.g., check numvfs == 0 if enabling VFs, ensure
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) numvfs <= totalvfs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) The second method is the recommended method for new/future VF devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) How can I use the Virtual Functions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) -----------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) The VF is treated as hot-plugged PCI devices in the kernel, so they
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) should be able to work in the same way as real PCI devices. The VF
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) requires device driver that is same as a normal PCI device's.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) Developer Guide
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) ===============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) SR-IOV API
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) ----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) To enable SR-IOV capability:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) (a) For the first method, in the driver::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) 'nr_virtfn' is number of VFs to be enabled.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) (b) For the second method, from sysfs::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) echo 'nr_virtfn' > \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79) To disable SR-IOV capability:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) (a) For the first method, in the driver::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) void pci_disable_sriov(struct pci_dev *dev);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) (b) For the second method, from sysfs::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) echo 0 > \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) To enable auto probing VFs by a compatible driver on the host, run
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) command below before enabling SR-IOV capabilities. This is the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) default behavior.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) echo 1 > \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_drivers_autoprobe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) To disable auto probing VFs by a compatible driver on the host, run
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) command below before enabling SR-IOV capabilities. Updating this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) entry will not affect VFs which are already probed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) echo 0 > \
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_drivers_autoprobe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) Usage example
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) -------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) Following piece of code illustrates the usage of the SR-IOV API.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) static int dev_probe(struct pci_dev *dev, const struct pci_device_id *id)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) pci_enable_sriov(dev, NR_VIRTFN);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) ...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) return 0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) static void dev_remove(struct pci_dev *dev)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) pci_disable_sriov(dev);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) ...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) static int dev_suspend(struct pci_dev *dev, pm_message_t state)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) ...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) return 0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) static int dev_resume(struct pci_dev *dev)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) ...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) return 0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) static void dev_shutdown(struct pci_dev *dev)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) ...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) static int dev_sriov_configure(struct pci_dev *dev, int numvfs)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) if (numvfs > 0) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) ...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) pci_enable_sriov(dev, numvfs);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) ...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) return numvfs;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) if (numvfs == 0) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) ....
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) pci_disable_sriov(dev);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) ...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) return 0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) static struct pci_driver dev_driver = {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) .name = "SR-IOV Physical Function driver",
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) .id_table = dev_id_table,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) .probe = dev_probe,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) .remove = dev_remove,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) .suspend = dev_suspend,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169) .resume = dev_resume,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) .shutdown = dev_shutdown,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) .sriov_configure = dev_sriov_configure,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) };
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags