^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) .. include:: <isonum.txt>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) ==========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) The MSI Driver Guide HOWTO
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) ==========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) :Authors: Tom L Nguyen; Martine Silbermann; Matthew Wilcox
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) :Copyright: 2003, 2008 Intel Corporation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) About this guide
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) ================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) This guide describes the basics of Message Signaled Interrupts (MSIs),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) the advantages of using MSI over traditional interrupt mechanisms, how
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) to change your driver to use MSI or MSI-X and some basic diagnostics to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) try if a device doesn't support MSIs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) What are MSIs?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) ==============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) A Message Signaled Interrupt is a write from the device to a special
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) address which causes an interrupt to be received by the CPU.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) The MSI capability was first specified in PCI 2.2 and was later enhanced
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) in PCI 3.0 to allow each interrupt to be masked individually. The MSI-X
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) capability was also introduced with PCI 3.0. It supports more interrupts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) per device than MSI and allows interrupts to be independently configured.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) Devices may support both MSI and MSI-X, but only one can be enabled at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) a time.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) Why use MSIs?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) =============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) There are three reasons why using MSIs can give an advantage over
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) traditional pin-based interrupts.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) Pin-based PCI interrupts are often shared amongst several devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) To support this, the kernel must call each interrupt handler associated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) with an interrupt, which leads to reduced performance for the system as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) a whole. MSIs are never shared, so this problem cannot arise.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) When a device writes data to memory, then raises a pin-based interrupt,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) it is possible that the interrupt may arrive before all the data has
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) arrived in memory (this becomes more likely with devices behind PCI-PCI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) bridges). In order to ensure that all the data has arrived in memory,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) the interrupt handler must read a register on the device which raised
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) the interrupt. PCI transaction ordering rules require that all the data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) arrive in memory before the value may be returned from the register.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) Using MSIs avoids this problem as the interrupt-generating write cannot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) pass the data writes, so by the time the interrupt is raised, the driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) knows that all the data has arrived in memory.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) PCI devices can only support a single pin-based interrupt per function.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) Often drivers have to query the device to find out what event has
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) occurred, slowing down interrupt handling for the common case. With
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) MSIs, a device can support more interrupts, allowing each interrupt
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) to be specialised to a different purpose. One possible design gives
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) infrequent conditions (such as errors) their own interrupt which allows
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) the driver to handle the normal interrupt handling path more efficiently.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) Other possible designs include giving one interrupt to each packet queue
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) in a network card or each port in a storage controller.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) How to use MSIs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) ===============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) PCI devices are initialised to use pin-based interrupts. The device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) driver has to set up the device to use MSI or MSI-X. Not all machines
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) support MSIs correctly, and for those machines, the APIs described below
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) will simply fail and the device will continue to use pin-based interrupts.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) Include kernel support for MSIs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) -------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) To support MSI or MSI-X, the kernel must be built with the CONFIG_PCI_MSI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) option enabled. This option is only available on some architectures,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) and it may depend on some other options also being set. For example,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) on x86, you must also enable X86_UP_APIC or SMP in order to see the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) CONFIG_PCI_MSI option.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) Using MSI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) ---------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) Most of the hard work is done for the driver in the PCI layer. The driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) simply has to request that the PCI layer set up the MSI capability for this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) To automatically use MSI or MSI-X interrupt vectors, use the following
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) function::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) int pci_alloc_irq_vectors(struct pci_dev *dev, unsigned int min_vecs,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) unsigned int max_vecs, unsigned int flags);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) which allocates up to max_vecs interrupt vectors for a PCI device. It
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) returns the number of vectors allocated or a negative error. If the device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) has a requirements for a minimum number of vectors the driver can pass a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) min_vecs argument set to this limit, and the PCI core will return -ENOSPC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) if it can't meet the minimum number of vectors.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) The flags argument is used to specify which type of interrupt can be used
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) by the device and the driver (PCI_IRQ_LEGACY, PCI_IRQ_MSI, PCI_IRQ_MSIX).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) A convenient short-hand (PCI_IRQ_ALL_TYPES) is also available to ask for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) any possible kind of interrupt. If the PCI_IRQ_AFFINITY flag is set,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) pci_alloc_irq_vectors() will spread the interrupts around the available CPUs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) To get the Linux IRQ numbers passed to request_irq() and free_irq() and the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) vectors, use the following function::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) int pci_irq_vector(struct pci_dev *dev, unsigned int nr);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) Any allocated resources should be freed before removing the device using
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) the following function::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) void pci_free_irq_vectors(struct pci_dev *dev);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) If a device supports both MSI-X and MSI capabilities, this API will use the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) MSI-X facilities in preference to the MSI facilities. MSI-X supports any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) number of interrupts between 1 and 2048. In contrast, MSI is restricted to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) a maximum of 32 interrupts (and must be a power of two). In addition, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) MSI interrupt vectors must be allocated consecutively, so the system might
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) not be able to allocate as many vectors for MSI as it could for MSI-X. On
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) some platforms, MSI interrupts must all be targeted at the same set of CPUs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) whereas MSI-X interrupts can all be targeted at different CPUs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) If a device supports neither MSI-X or MSI it will fall back to a single
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) legacy IRQ vector.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) The typical usage of MSI or MSI-X interrupts is to allocate as many vectors
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) as possible, likely up to the limit supported by the device. If nvec is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) larger than the number supported by the device it will automatically be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) capped to the supported limit, so there is no need to query the number of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) vectors supported beforehand::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) nvec = pci_alloc_irq_vectors(pdev, 1, nvec, PCI_IRQ_ALL_TYPES)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) if (nvec < 0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) goto out_err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) If a driver is unable or unwilling to deal with a variable number of MSI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) interrupts it can request a particular number of interrupts by passing that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) number to pci_alloc_irq_vectors() function as both 'min_vecs' and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) 'max_vecs' parameters::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) ret = pci_alloc_irq_vectors(pdev, nvec, nvec, PCI_IRQ_ALL_TYPES);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) if (ret < 0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) goto out_err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) The most notorious example of the request type described above is enabling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) the single MSI mode for a device. It could be done by passing two 1s as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) 'min_vecs' and 'max_vecs'::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) ret = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_ALL_TYPES);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) if (ret < 0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) goto out_err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) Some devices might not support using legacy line interrupts, in which case
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) the driver can specify that only MSI or MSI-X is acceptable::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) nvec = pci_alloc_irq_vectors(pdev, 1, nvec, PCI_IRQ_MSI | PCI_IRQ_MSIX);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) if (nvec < 0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) goto out_err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) Legacy APIs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) -----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) The following old APIs to enable and disable MSI or MSI-X interrupts should
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) not be used in new code::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) pci_enable_msi() /* deprecated */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) pci_disable_msi() /* deprecated */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) pci_enable_msix_range() /* deprecated */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) pci_enable_msix_exact() /* deprecated */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) pci_disable_msix() /* deprecated */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) Additionally there are APIs to provide the number of supported MSI or MSI-X
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) vectors: pci_msi_vec_count() and pci_msix_vec_count(). In general these
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) should be avoided in favor of letting pci_alloc_irq_vectors() cap the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) number of vectors. If you have a legitimate special use case for the count
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) of vectors we might have to revisit that decision and add a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) pci_nr_irq_vectors() helper that handles MSI and MSI-X transparently.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) Considerations when using MSIs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) ------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) Spinlocks
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) ~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) Most device drivers have a per-device spinlock which is taken in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) interrupt handler. With pin-based interrupts or a single MSI, it is not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) necessary to disable interrupts (Linux guarantees the same interrupt will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) not be re-entered). If a device uses multiple interrupts, the driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) must disable interrupts while the lock is held. If the device sends
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) a different interrupt, the driver will deadlock trying to recursively
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) acquire the spinlock. Such deadlocks can be avoided by using
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) spin_lock_irqsave() or spin_lock_irq() which disable local interrupts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) and acquire the lock (see Documentation/kernel-hacking/locking.rst).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) How to tell whether MSI/MSI-X is enabled on a device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) ----------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) Using 'lspci -v' (as root) may show some devices with "MSI", "Message
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) Signalled Interrupts" or "MSI-X" capabilities. Each of these capabilities
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) has an 'Enable' flag which is followed with either "+" (enabled)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) or "-" (disabled).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) MSI quirks
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) ==========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) Several PCI chipsets or devices are known not to support MSIs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) The PCI stack provides three ways to disable MSIs:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) 1. globally
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) 2. on all devices behind a specific bridge
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) 3. on a single device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) Disabling MSIs globally
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222) -----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224) Some host chipsets simply don't support MSIs properly. If we're
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) lucky, the manufacturer knows this and has indicated it in the ACPI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) FADT table. In this case, Linux automatically disables MSIs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227) Some boards don't include this information in the table and so we have
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) to detect them ourselves. The complete list of these is found near the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229) quirk_disable_all_msi() function in drivers/pci/quirks.c.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231) If you have a board which has problems with MSIs, you can pass pci=nomsi
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) on the kernel command line to disable MSIs on all devices. It would be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233) in your best interests to report the problem to linux-pci@vger.kernel.org
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234) including a full 'lspci -v' so we can add the quirks to the kernel.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) Disabling MSIs below a bridge
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) -----------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) Some PCI bridges are not able to route MSIs between busses properly.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240) In this case, MSIs must be disabled on all devices behind the bridge.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) Some bridges allow you to enable MSIs by changing some bits in their
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243) PCI configuration space (especially the Hypertransport chipsets such
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) as the nVidia nForce and Serverworks HT2000). As with host chipsets,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) Linux mostly knows about them and automatically enables MSIs if it can.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) If you have a bridge unknown to Linux, you can enable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247) MSIs in configuration space using whatever method you know works, then
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) enable MSIs on that bridge by doing::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250) echo 1 > /sys/bus/pci/devices/$bridge/msi_bus
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252) where $bridge is the PCI address of the bridge you've enabled (eg
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) 0000:00:0e.0).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255) To disable MSIs, echo 0 instead of 1. Changing this value should be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) done with caution as it could break interrupt handling for all devices
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) below this bridge.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) Again, please notify linux-pci@vger.kernel.org of any bridges that need
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260) special handling.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262) Disabling MSIs on a single device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263) ---------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265) Some devices are known to have faulty MSI implementations. Usually this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266) is handled in the individual device driver, but occasionally it's necessary
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267) to handle this with a quirk. Some drivers have an option to disable use
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268) of MSI. While this is a convenient workaround for the driver author,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269) it is not good practice, and should not be emulated.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271) Finding why MSIs are disabled on a device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272) -----------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274) From the above three sections, you can see that there are many reasons
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275) why MSIs may not be enabled for a given device. Your first step should
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276) be to examine your dmesg carefully to determine whether MSIs are enabled
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277) for your machine. You should also check your .config to be sure you
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278) have enabled CONFIG_PCI_MSI.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280) Then, 'lspci -t' gives the list of bridges above a device. Reading
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281) `/sys/bus/pci/devices/*/msi_bus` will tell you whether MSIs are enabled (1)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282) or disabled (0). If 0 is found in any of the msi_bus files belonging
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283) to bridges between the PCI root and the device, MSIs are disabled.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285) It is also worth checking the device driver to see whether it supports MSIs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286) For example, it may contain calls to pci_alloc_irq_vectors() with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287) PCI_IRQ_MSI or PCI_IRQ_MSIX flags.
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags