^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) ===========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) NTB Drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) ===========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) NTB (Non-Transparent Bridge) is a type of PCI-Express bridge chip that connects
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) the separate memory systems of two or more computers to the same PCI-Express
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) fabric. Existing NTB hardware supports a common feature set: doorbell
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) registers and memory translation windows, as well as non common features like
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) scratchpad and message registers. Scratchpad registers are read-and-writable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) registers that are accessible from either side of the device, so that peers can
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) exchange a small amount of information at a fixed address. Message registers can
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) be utilized for the same purpose. Additionally they are provided with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) special status bits to make sure the information isn't rewritten by another
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) peer. Doorbell registers provide a way for peers to send interrupt events.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) Memory windows allow translated read and write access to the peer memory.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) NTB Core Driver (ntb)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) =====================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) The NTB core driver defines an api wrapping the common feature set, and allows
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) clients interested in NTB features to discover NTB the devices supported by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) hardware drivers. The term "client" is used here to mean an upper layer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) component making use of the NTB api. The term "driver," or "hardware driver,"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) is used here to mean a driver for a specific vendor and model of NTB hardware.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) NTB Client Drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) ==================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) NTB client drivers should register with the NTB core driver. After
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) registering, the client probe and remove functions will be called appropriately
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) as ntb hardware, or hardware drivers, are inserted and removed. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) registration uses the Linux Device framework, so it should feel familiar to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) anyone who has written a pci driver.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) NTB Typical client driver implementation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) ----------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) Primary purpose of NTB is to share some peace of memory between at least two
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) systems. So the NTB device features like Scratchpad/Message registers are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) mainly used to perform the proper memory window initialization. Typically
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) there are two types of memory window interfaces supported by the NTB API:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) inbound translation configured on the local ntb port and outbound translation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) configured by the peer, on the peer ntb port. The first type is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) depicted on the next figure::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46) Inbound translation:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) Memory: Local NTB Port: Peer NTB Port: Peer MMIO:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) ____________
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) | dma-mapped |-ntb_mw_set_trans(addr) |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) | memory | _v____________ | ______________
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) | (addr) |<======| MW xlat addr |<====| MW base addr |<== memory-mapped IO
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) |------------| |--------------| | |--------------|
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) So typical scenario of the first type memory window initialization looks:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) 1) allocate a memory region, 2) put translated address to NTB config,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) 3) somehow notify a peer device of performed initialization, 4) peer device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) maps corresponding outbound memory window so to have access to the shared
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) memory region.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) The second type of interface, that implies the shared windows being
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) initialized by a peer device, is depicted on the figure::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) Outbound translation:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) Memory: Local NTB Port: Peer NTB Port: Peer MMIO:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) ____________ ______________
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) | dma-mapped | | | MW base addr |<== memory-mapped IO
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) | memory | | |--------------|
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) | (addr) |<===================| MW xlat addr |<-ntb_peer_mw_set_trans(addr)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) |------------| | |--------------|
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) Typical scenario of the second type interface initialization would be:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) 1) allocate a memory region, 2) somehow deliver a translated address to a peer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) device, 3) peer puts the translated address to NTB config, 4) peer device maps
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) outbound memory window so to have access to the shared memory region.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) As one can see the described scenarios can be combined in one portable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79) algorithm.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) Local device:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) 1) Allocate memory for a shared window
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) 2) Initialize memory window by translated address of the allocated region
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) (it may fail if local memory window initialization is unsupported)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) 3) Send the translated address and memory window index to a peer device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) Peer device:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) 1) Initialize memory window with retrieved address of the allocated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) by another device memory region (it may fail if peer memory window
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) initialization is unsupported)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) 2) Map outbound memory window
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) In accordance with this scenario, the NTB Memory Window API can be used as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) follows:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) Local device:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) 1) ntb_mw_count(pidx) - retrieve number of memory ranges, which can
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) be allocated for memory windows between local device and peer device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) of port with specified index.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) 2) ntb_get_align(pidx, midx) - retrieve parameters restricting the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) shared memory region alignment and size. Then memory can be properly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) allocated.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) 3) Allocate physically contiguous memory region in compliance with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) restrictions retrieved in 2).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) 4) ntb_mw_set_trans(pidx, midx) - try to set translation address of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) the memory window with specified index for the defined peer device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) (it may fail if local translated address setting is not supported)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) 5) Send translated base address (usually together with memory window
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) number) to the peer device using, for instance, scratchpad or message
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) registers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) Peer device:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) 1) ntb_peer_mw_set_trans(pidx, midx) - try to set received from other
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) device (related to pidx) translated address for specified memory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) window. It may fail if retrieved address, for instance, exceeds
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) maximum possible address or isn't properly aligned.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) 2) ntb_peer_mw_get_addr(widx) - retrieve MMIO address to map the memory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) window so to have an access to the shared memory.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) Also it is worth to note, that method ntb_mw_count(pidx) should return the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) same value as ntb_peer_mw_count() on the peer with port index - pidx.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) NTB Transport Client (ntb\_transport) and NTB Netdev (ntb\_netdev)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) ------------------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) The primary client for NTB is the Transport client, used in tandem with NTB
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) Netdev. These drivers function together to create a logical link to the peer,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) across the ntb, to exchange packets of network data. The Transport client
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) establishes a logical link to the peer, and creates queue pairs to exchange
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) messages and data. The NTB Netdev then creates an ethernet device using a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) Transport queue pair. Network data is copied between socket buffers and the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) Transport queue pair buffer. The Transport client may be used for other things
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) besides Netdev, however no other applications have yet been written.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) NTB Ping Pong Test Client (ntb\_pingpong)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) -----------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) The Ping Pong test client serves as a demonstration to exercise the doorbell
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) and scratchpad registers of NTB hardware, and as an example simple NTB client.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) Ping Pong enables the link when started, waits for the NTB link to come up, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) then proceeds to read and write the doorbell scratchpad registers of the NTB.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) The peers interrupt each other using a bit mask of doorbell bits, which is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) shifted by one in each round, to test the behavior of multiple doorbell bits
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) and interrupt vectors. The Ping Pong driver also reads the first local
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) scratchpad, and writes the value plus one to the first peer scratchpad, each
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) round before writing the peer doorbell register.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) Module Parameters:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) * unsafe - Some hardware has known issues with scratchpad and doorbell
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) registers. By default, Ping Pong will not attempt to exercise such
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) hardware. You may override this behavior at your own risk by setting
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) unsafe=1.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) * delay\_ms - Specify the delay between receiving a doorbell
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) interrupt event and setting the peer doorbell register for the next
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) round.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) * init\_db - Specify the doorbell bits to start new series of rounds. A new
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) series begins once all the doorbell bits have been shifted out of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) range.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) * dyndbg - It is suggested to specify dyndbg=+p when loading this module, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) then to observe debugging output on the console.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) NTB Tool Test Client (ntb\_tool)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) --------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) The Tool test client serves for debugging, primarily, ntb hardware and drivers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) The Tool provides access through debugfs for reading, setting, and clearing the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) NTB doorbell, and reading and writing scratchpads.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) The Tool does not currently have any module parameters.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) Debugfs Files:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) * *debugfs*/ntb\_tool/*hw*/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) A directory in debugfs will be created for each
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) NTB device probed by the tool. This directory is shortened to *hw*
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) * *hw*/db
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) This file is used to read, set, and clear the local doorbell. Not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) all operations may be supported by all hardware. To read the doorbell,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) read the file. To set the doorbell, write `s` followed by the bits to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) set (eg: `echo 's 0x0101' > db`). To clear the doorbell, write `c`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) followed by the bits to clear.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) * *hw*/mask
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) This file is used to read, set, and clear the local doorbell mask.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) See *db* for details.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) * *hw*/peer\_db
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) This file is used to read, set, and clear the peer doorbell.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) See *db* for details.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) * *hw*/peer\_mask
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) This file is used to read, set, and clear the peer doorbell
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) mask. See *db* for details.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) * *hw*/spad
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) This file is used to read and write local scratchpads. To read
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) the values of all scratchpads, read the file. To write values, write a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) series of pairs of scratchpad number and value
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) (eg: `echo '4 0x123 7 0xabc' > spad`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) # to set scratchpads `4` and `7` to `0x123` and `0xabc`, respectively).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) * *hw*/peer\_spad
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) This file is used to read and write peer scratchpads. See
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) *spad* for details.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) NTB MSI Test Client (ntb\_msi\_test)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) ------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) The MSI test client serves to test and debug the MSI library which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) allows for passing MSI interrupts across NTB memory windows. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) test client is interacted with through the debugfs filesystem:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210) * *debugfs*/ntb\_tool/*hw*/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) A directory in debugfs will be created for each
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) NTB device probed by the tool. This directory is shortened to *hw*
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) * *hw*/port
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) This file describes the local port number
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) * *hw*/irq*_occurrences
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) One occurrences file exists for each interrupt and, when read,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) returns the number of times the interrupt has been triggered.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) * *hw*/peer*/port
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) This file describes the port number for each peer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) * *hw*/peer*/count
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222) This file describes the number of interrupts that can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223) triggered on each peer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224) * *hw*/peer*/trigger
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) Writing an interrupt number (any number less than the value
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) specified in count) will trigger the interrupt on the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227) specified peer. That peer's interrupt's occurrence file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) should be incremented.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230) NTB Hardware Drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231) ====================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233) NTB hardware drivers should register devices with the NTB core driver. After
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234) registering, clients probe and remove functions will be called.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) NTB Intel Hardware Driver (ntb\_hw\_intel)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) ------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) The Intel hardware driver supports NTB on Xeon and Atom CPUs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241) Module Parameters:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243) * b2b\_mw\_idx
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) If the peer ntb is to be accessed via a memory window, then use
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) this memory window to access the peer ntb. A value of zero or positive
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) starts from the first mw idx, and a negative value starts from the last
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247) mw idx. Both sides MUST set the same value here! The default value is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) `-1`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249) * b2b\_mw\_share
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250) If the peer ntb is to be accessed via a memory window, and if
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251) the memory window is large enough, still allow the client to use the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252) second half of the memory window for address translation to the peer.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) * xeon\_b2b\_usd\_bar2\_addr64
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254) If using B2B topology on Xeon hardware, use
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255) this 64 bit address on the bus between the NTB devices for the window
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) at BAR2, on the upstream side of the link.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) * xeon\_b2b\_usd\_bar4\_addr64 - See *xeon\_b2b\_bar2\_addr64*.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) * xeon\_b2b\_usd\_bar4\_addr32 - See *xeon\_b2b\_bar2\_addr64*.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) * xeon\_b2b\_usd\_bar5\_addr32 - See *xeon\_b2b\_bar2\_addr64*.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260) * xeon\_b2b\_dsd\_bar2\_addr64 - See *xeon\_b2b\_bar2\_addr64*.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261) * xeon\_b2b\_dsd\_bar4\_addr64 - See *xeon\_b2b\_bar2\_addr64*.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262) * xeon\_b2b\_dsd\_bar4\_addr32 - See *xeon\_b2b\_bar2\_addr64*.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263) * xeon\_b2b\_dsd\_bar5\_addr32 - See *xeon\_b2b\_bar2\_addr64*.
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags