Orange Pi5 kernel

^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   1) ============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   2) Introduction
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   3) ============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   4) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   5) The RapidIO standard is a packet-based fabric interconnect standard designed for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   6) use in embedded systems. Development of the RapidIO standard is directed by the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   7) RapidIO Trade Association (RTA). The current version of the RapidIO specification
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   8) is publicly available for download from the RTA web-site [1].
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   9) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  10) This document describes the basics of the Linux RapidIO subsystem and provides
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  11) information on its major components.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  12) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  13) 1 Overview
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  14) ==========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  15) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  16) Because the RapidIO subsystem follows the Linux device model it is integrated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  17) into the kernel similarly to other buses by defining RapidIO-specific device and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  18) bus types and registering them within the device model.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  19) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  20) The Linux RapidIO subsystem is architecture independent and therefore defines
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  21) architecture-specific interfaces that provide support for common RapidIO
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  22) subsystem operations.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  23) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  24) 2. Core Components
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  25) ==================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  26) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  27) A typical RapidIO network is a combination of endpoints and switches.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  28) Each of these components is represented in the subsystem by an associated data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  29) structure. The core logical components of the RapidIO subsystem are defined
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  30) in include/linux/rio.h file.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  31) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  32) 2.1 Master Port
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  33) ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  34) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  35) A master port (or mport) is a RapidIO interface controller that is local to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  36) processor executing the Linux code. A master port generates and receives RapidIO
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  37) packets (transactions). In the RapidIO subsystem each master port is represented
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  38) by a rio_mport data structure. This structure contains master port specific
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  39) resources such as mailboxes and doorbells. The rio_mport also includes a unique
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  40) host device ID that is valid when a master port is configured as an enumerating
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  41) host.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  42) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  43) RapidIO master ports are serviced by subsystem specific mport device drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  44) that provide functionality defined for this subsystem. To provide a hardware
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  45) independent interface for RapidIO subsystem operations, rio_mport structure
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  46) includes rio_ops data structure which contains pointers to hardware specific
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  47) implementations of RapidIO functions.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  48) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  49) 2.2 Device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  50) ----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  51) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  52) A RapidIO device is any endpoint (other than mport) or switch in the network.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  53) All devices are presented in the RapidIO subsystem by corresponding rio_dev data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  54) structure. Devices form one global device list and per-network device lists
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  55) (depending on number of available mports and networks).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  56) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  57) 2.3 Switch
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  58) ----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  59) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  60) A RapidIO switch is a special class of device that routes packets between its
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  61) ports towards their final destination. The packet destination port within a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  62) switch is defined by an internal routing table. A switch is presented in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  63) RapidIO subsystem by rio_dev data structure expanded by additional rio_switch
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  64) data structure, which contains switch specific information such as copy of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  65) routing table and pointers to switch specific functions.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  66) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  67) The RapidIO subsystem defines the format and initialization method for subsystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  68) specific switch drivers that are designed to provide hardware-specific
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  69) implementation of common switch management routines.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  70) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  71) 2.4 Network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  72) -----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  73) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  74) A RapidIO network is a combination of interconnected endpoint and switch devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  75) Each RapidIO network known to the system is represented by corresponding rio_net
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  76) data structure. This structure includes lists of all devices and local master
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  77) ports that form the same network. It also contains a pointer to the default
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  78) master port that is used to communicate with devices within the network.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  79) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  80) 2.5 Device Drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  81) ------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  82) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  83) RapidIO device-specific drivers follow Linux Kernel Driver Model and are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  84) intended to support specific RapidIO devices attached to the RapidIO network.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  85) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  86) 2.6 Subsystem Interfaces
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  87) ------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  88) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  89) RapidIO interconnect specification defines features that may be used to provide
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  90) one or more common service layers for all participating RapidIO devices. These
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  91) common services may act separately from device-specific drivers or be used by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  92) device-specific drivers. Example of such service provider is the RIONET driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  93) which implements Ethernet-over-RapidIO interface. Because only one driver can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  94) registered for a device, all common RapidIO services have to be registered as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  95) subsystem interfaces. This allows to have multiple common services attached to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  96) the same device without blocking attachment of a device-specific driver.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  97) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  98) 3. Subsystem Initialization
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  99) ===========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) In order to initialize the RapidIO subsystem, a platform must initialize and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) register at least one master port within the RapidIO network. To register mport
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) within the subsystem controller driver's initialization code calls function
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) rio_register_mport() for each available master port.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) After all active master ports are registered with a RapidIO subsystem,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) an enumeration and/or discovery routine may be called automatically or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) by user-space command.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) RapidIO subsystem can be configured to be built as a statically linked or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) modular component of the kernel (see details below).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) 4. Enumeration and Discovery
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) ============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) 4.1 Overview
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) ------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) RapidIO subsystem configuration options allow users to build enumeration and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) discovery methods as statically linked components or loadable modules.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) An enumeration/discovery method implementation and available input parameters
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) define how any given method can be attached to available RapidIO mports:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) simply to all available mports OR individually to the specified mport device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) Depending on selected enumeration/discovery build configuration, there are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) several methods to initiate an enumeration and/or discovery process:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128)   (a) Statically linked enumeration and discovery process can be started
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)   automatically during kernel initialization time using corresponding module
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130)   parameters. This was the original method used since introduction of RapidIO
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)   subsystem. Now this method relies on enumerator module parameter which is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132)   'rio-scan.scan' for existing basic enumeration/discovery method.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133)   When automatic start of enumeration/discovery is used a user has to ensure
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134)   that all discovering endpoints are started before the enumerating endpoint
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135)   and are waiting for enumeration to be completed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136)   Configuration option CONFIG_RAPIDIO_DISC_TIMEOUT defines time that discovering
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137)   endpoint waits for enumeration to be completed. If the specified timeout
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)   expires the discovery process is terminated without obtaining RapidIO network
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)   information. NOTE: a timed out discovery process may be restarted later using
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140)   a user-space command as it is described below (if the given endpoint was
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)   enumerated successfully).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143)   (b) Statically linked enumeration and discovery process can be started by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144)   a command from user space. This initiation method provides more flexibility
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145)   for a system startup compared to the option (a) above. After all participating
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146)   endpoints have been successfully booted, an enumeration process shall be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)   started first by issuing a user-space command, after an enumeration is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148)   completed a discovery process can be started on all remaining endpoints.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150)   (c) Modular enumeration and discovery process can be started by a command from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151)   user space. After an enumeration/discovery module is loaded, a network scan
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152)   process can be started by issuing a user-space command.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153)   Similar to the option (b) above, an enumerator has to be started first.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155)   (d) Modular enumeration and discovery process can be started by a module
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156)   initialization routine. In this case an enumerating module shall be loaded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157)   first.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) When a network scan process is started it calls an enumeration or discovery
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) routine depending on the configured role of a master port: host or agent.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) Enumeration is performed by a master port if it is configured as a host port by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) assigning a host destination ID greater than or equal to zero. The host
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) destination ID can be assigned to a master port using various methods depending
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) on RapidIO subsystem build configuration:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)   (a) For a statically linked RapidIO subsystem core use command line parameter
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168)   "rapidio.hdid=" with a list of destination ID assignments in order of mport
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)   device registration. For example, in a system with two RapidIO controllers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170)   the command line parameter "rapidio.hdid=-1,7" will result in assignment of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171)   the host destination ID=7 to the second RapidIO controller, while the first
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172)   one will be assigned destination ID=-1.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174)   (b) If the RapidIO subsystem core is built as a loadable module, in addition
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175)   to the method shown above, the host destination ID(s) can be specified using
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176)   traditional methods of passing module parameter "hdid=" during its loading:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178)   - from command line: "modprobe rapidio hdid=-1,7", or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179)   - from modprobe configuration file using configuration command "options",
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180)     like in this example: "options rapidio hdid=-1,7". An example of modprobe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181)     configuration file is provided in the section below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) NOTES:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184)   (i) if "hdid=" parameter is omitted all available mport will be assigned
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185)   destination ID = -1;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187)   (ii) the "hdid=" parameter in systems with multiple mports can have
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188)   destination ID assignments omitted from the end of list (default = -1).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) If the host device ID for a specific master port is set to -1, the discovery
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) process will be performed for it.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) The enumeration and discovery routines use RapidIO maintenance transactions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) to access the configuration space of devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) NOTE: If RapidIO switch-specific device drivers are built as loadable modules
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) they must be loaded before enumeration/discovery process starts.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) This requirement is cased by the fact that enumeration/discovery methods invoke
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) vendor-specific callbacks on early stages.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) 4.2 Automatic Start of Enumeration and Discovery
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) ------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) Automatic enumeration/discovery start method is applicable only to built-in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) enumeration/discovery RapidIO configuration selection. To enable automatic
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) enumeration/discovery start by existing basic enumerator method set use boot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) command line parameter "rio-scan.scan=1".
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) This configuration requires synchronized start of all RapidIO endpoints that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210) form a network which will be enumerated/discovered. Discovering endpoints have
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) to be started before an enumeration starts to ensure that all RapidIO
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) controllers have been initialized and are ready to be discovered. Configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) parameter CONFIG_RAPIDIO_DISC_TIMEOUT defines time (in seconds) which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) a discovering endpoint will wait for enumeration to be completed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) When automatic enumeration/discovery start is selected, basic method's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) initialization routine calls rio_init_mports() to perform enumeration or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) discovery for all known mport devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) Depending on RapidIO network size and configuration this automatic
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) enumeration/discovery start method may be difficult to use due to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222) requirement for synchronized start of all endpoints.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224) 4.3 User-space Start of Enumeration and Discovery
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) -------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227) User-space start of enumeration and discovery can be used with built-in and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) modular build configurations. For user-space controlled start RapidIO subsystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229) creates the sysfs write-only attribute file '/sys/bus/rapidio/scan'. To initiate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230) an enumeration or discovery process on specific mport device, a user needs to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231) write mport_ID (not RapidIO destination ID) into that file. The mport_ID is a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) sequential number (0 ... RIO_MAX_MPORTS) assigned during mport device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233) registration. For example for machine with single RapidIO controller, mport_ID
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234) for that controller always will be 0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) To initiate RapidIO enumeration/discovery on all available mports a user may
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) write '-1' (or RIO_MPORT_ANY) into the scan attribute file.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) 4.4 Basic Enumeration Method
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240) ----------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) This is an original enumeration/discovery method which is available since
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243) first release of RapidIO subsystem code. The enumeration process is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) implemented according to the enumeration algorithm outlined in the RapidIO
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) Interconnect Specification: Annex I [1].
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247) This method can be configured as statically linked or loadable module.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) The method's single parameter "scan" allows to trigger the enumeration/discovery
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249) process from module initialization routine.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251) This enumeration/discovery method can be started only once and does not support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252) unloading if it is built as a module.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254) The enumeration process traverses the network using a recursive depth-first
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255) algorithm. When a new device is found, the enumerator takes ownership of that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) device by writing into the Host Device ID Lock CSR. It does this to ensure that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) the enumerator has exclusive right to enumerate the device. If device ownership
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) is successfully acquired, the enumerator allocates a new rio_dev structure and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) initializes it according to device capabilities.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261) If the device is an endpoint, a unique device ID is assigned to it and its value
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262) is written into the device's Base Device ID CSR.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264) If the device is a switch, the enumerator allocates an additional rio_switch
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265) structure to store switch specific information. Then the switch's vendor ID and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266) device ID are queried against a table of known RapidIO switches. Each switch
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267) table entry contains a pointer to a switch-specific initialization routine that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268) initializes pointers to the rest of switch specific operations, and performs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269) hardware initialization if necessary. A RapidIO switch does not have a unique
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270) device ID; it relies on hopcount and routing for device ID of an attached
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271) endpoint if access to its configuration registers is required. If a switch (or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272) chain of switches) does not have any endpoint (except enumerator) attached to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273) it, a fake device ID will be assigned to configure a route to that switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274) In the case of a chain of switches without endpoint, one fake device ID is used
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275) to configure a route through the entire chain and switches are differentiated by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276) their hopcount value.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278) For both endpoints and switches the enumerator writes a unique component tag
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279) into device's Component Tag CSR. That unique value is used by the error
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280) management notification mechanism to identify a device that is reporting an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281) error management event.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283) Enumeration beyond a switch is completed by iterating over each active egress
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284) port of that switch. For each active link, a route to a default device ID
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285) (0xFF for 8-bit systems and 0xFFFF for 16-bit systems) is temporarily written
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286) into the routing table. The algorithm recurs by calling itself with hopcount + 1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287) and the default device ID in order to access the device on the active port.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 288) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 289) After the host has completed enumeration of the entire network it releases
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 290) devices by clearing device ID locks (calls rio_clear_locks()). For each endpoint
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 291) in the system, it sets the Discovered bit in the Port General Control CSR
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 292) to indicate that enumeration is completed and agents are allowed to execute
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 293) passive discovery of the network.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 294) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 295) The discovery process is performed by agents and is similar to the enumeration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 296) process that is described above. However, the discovery process is performed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 297) without changes to the existing routing because agents only gather information
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 298) about RapidIO network structure and are building an internal map of discovered
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 299) devices. This way each Linux-based component of the RapidIO subsystem has
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 300) a complete view of the network. The discovery process can be performed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 301) simultaneously by several agents. After initializing its RapidIO master port
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 302) each agent waits for enumeration completion by the host for the configured wait
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 303) time period. If this wait time period expires before enumeration is completed,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 304) an agent skips RapidIO discovery and continues with remaining kernel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 305) initialization.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 306) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 307) 4.5 Adding New Enumeration/Discovery Method
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 308) -------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 309) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 310) RapidIO subsystem code organization allows addition of new enumeration/discovery
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 311) methods as new configuration options without significant impact to the core
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 312) RapidIO code.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 313) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 314) A new enumeration/discovery method has to be attached to one or more mport
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 315) devices before an enumeration/discovery process can be started. Normally,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 316) method's module initialization routine calls rio_register_scan() to attach
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 317) an enumerator to a specified mport device (or devices). The basic enumerator
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 318) implementation demonstrates this process.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 319) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 320) 4.6 Using Loadable RapidIO Switch Drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 321) -----------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 322) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 323) In the case when RapidIO switch drivers are built as loadable modules a user
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 324) must ensure that they are loaded before the enumeration/discovery starts.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 325) This process can be automated by specifying pre- or post- dependencies in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 326) RapidIO-specific modprobe configuration file as shown in the example below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 327) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 328) File /etc/modprobe.d/rapidio.conf::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 329) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 330)   # Configure RapidIO subsystem modules
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 331) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 332)   # Set enumerator host destination ID (overrides kernel command line option)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 333)   options rapidio hdid=-1,2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 334) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 335)   # Load RapidIO switch drivers immediately after rapidio core module was loaded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 336)   softdep rapidio post: idt_gen2 idtcps tsi57x
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 337) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 338)   # OR :
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 339) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 340)   # Load RapidIO switch drivers just before rio-scan enumerator module is loaded
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 341)   softdep rio-scan pre: idt_gen2 idtcps tsi57x
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 342) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 343)   --------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 344) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 345) NOTE:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 346)   In the example above, one of "softdep" commands must be removed or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 347)   commented out to keep required module loading sequence.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 348) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 349) 5. References
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 350) =============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 351) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 352) [1] RapidIO Trade Association. RapidIO Interconnect Specifications.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 353)     http://www.rapidio.org.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 354) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 355) [2] Rapidio TA. Technology Comparisons.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 356)     http://www.rapidio.org/education/technology_comparisons/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 357) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 358) [3] RapidIO support for Linux.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 359)     https://lwn.net/Articles/139118/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 360) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 361) [4] Matt Porter. RapidIO for Linux. Ottawa Linux Symposium, 2005
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 362)     https://www.kernel.org/doc/ols/2005/ols2005v2-pages-43-56.pdf