^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) .. include:: <isonum.txt>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) ===========================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) The PCI Express Port Bus Driver Guide HOWTO
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) ===========================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) :Author: Tom L Nguyen tom.l.nguyen@intel.com 11/03/2004
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) :Copyright: |copy| 2004 Intel Corporation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) About this guide
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) ================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) This guide describes the basics of the PCI Express Port Bus driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) and provides information on how to enable the service drivers to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) register/unregister with the PCI Express Port Bus Driver.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) What is the PCI Express Port Bus Driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) =======================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) A PCI Express Port is a logical PCI-PCI Bridge structure. There
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) are two types of PCI Express Port: the Root Port and the Switch
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) Port. The Root Port originates a PCI Express link from a PCI Express
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) Root Complex and the Switch Port connects PCI Express links to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) internal logical PCI buses. The Switch Port, which has its secondary
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) bus representing the switch's internal routing logic, is called the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) switch's Upstream Port. The switch's Downstream Port is bridging from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) switch's internal routing bus to a bus representing the downstream
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) PCI Express link from the PCI Express Switch.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) A PCI Express Port can provide up to four distinct functions,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) referred to in this document as services, depending on its port type.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) PCI Express Port's services include native hotplug support (HP),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) power management event support (PME), advanced error reporting
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) support (AER), and virtual channel support (VC). These services may
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) be handled by a single complex driver or be individually distributed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) and handled by corresponding service drivers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) Why use the PCI Express Port Bus Driver?
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) ========================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) In existing Linux kernels, the Linux Device Driver Model allows a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) physical device to be handled by only a single driver. The PCI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) Express Port is a PCI-PCI Bridge device with multiple distinct
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46) services. To maintain a clean and simple solution each service
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) may have its own software service driver. In this case several
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) service drivers will compete for a single PCI-PCI Bridge device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) For example, if the PCI Express Root Port native hotplug service
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) driver is loaded first, it claims a PCI-PCI Bridge Root Port. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) kernel therefore does not load other service drivers for that Root
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) Port. In other words, it is impossible to have multiple service
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) drivers load and run on a PCI-PCI Bridge device simultaneously
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) using the current driver model.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) To enable multiple service drivers running simultaneously requires
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) having a PCI Express Port Bus driver, which manages all populated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) PCI Express Ports and distributes all provided service requests
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) to the corresponding service drivers as required. Some key
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) advantages of using the PCI Express Port Bus driver are listed below:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) - Allow multiple service drivers to run simultaneously on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) a PCI-PCI Bridge Port device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) - Allow service drivers implemented in an independent
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) staged approach.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) - Allow one service driver to run on multiple PCI-PCI Bridge
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) Port devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) - Manage and distribute resources of a PCI-PCI Bridge Port
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) device to requested service drivers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) Configuring the PCI Express Port Bus Driver vs. Service Drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) ===============================================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) Including the PCI Express Port Bus Driver Support into the Kernel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) -----------------------------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) Including the PCI Express Port Bus driver depends on whether the PCI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) Express support is included in the kernel config. The kernel will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) automatically include the PCI Express Port Bus driver as a kernel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) driver when the PCI Express support is enabled in the kernel.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) Enabling Service Driver Support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) -------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) PCI device drivers are implemented based on Linux Device Driver Model.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) All service drivers are PCI device drivers. As discussed above, it is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) impossible to load any service driver once the kernel has loaded the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) PCI Express Port Bus Driver. To meet the PCI Express Port Bus Driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) Model requires some minimal changes on existing service drivers that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) imposes no impact on the functionality of existing service drivers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) A service driver is required to use the two APIs shown below to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) register its service with the PCI Express Port Bus driver (see
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) section 5.2.1 & 5.2.2). It is important that a service driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) initializes the pcie_port_service_driver data structure, included in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) header file /include/linux/pcieport_if.h, before calling these APIs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) Failure to do so will result an identity mismatch, which prevents
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) the PCI Express Port Bus driver from loading a service driver.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) pcie_port_service_register
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) ~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) int pcie_port_service_register(struct pcie_port_service_driver *new)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) This API replaces the Linux Driver Model's pci_register_driver API. A
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) service driver should always calls pcie_port_service_register at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) module init. Note that after service driver being loaded, calls
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) such as pci_enable_device(dev) and pci_set_master(dev) are no longer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) necessary since these calls are executed by the PCI Port Bus driver.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) pcie_port_service_unregister
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) void pcie_port_service_unregister(struct pcie_port_service_driver *new)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) pcie_port_service_unregister replaces the Linux Driver Model's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) pci_unregister_driver. It's always called by service driver when a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) module exits.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) Sample Code
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) ~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) Below is sample service driver code to initialize the port service
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) driver data structure.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) static struct pcie_port_service_id service_id[] = { {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) .vendor = PCI_ANY_ID,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) .device = PCI_ANY_ID,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) .port_type = PCIE_RC_PORT,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) .service_type = PCIE_PORT_SERVICE_AER,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) }, { /* end: all zeroes */ }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) };
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) static struct pcie_port_service_driver root_aerdrv = {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) .name = (char *)device_name,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) .id_table = &service_id[0],
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) .probe = aerdrv_load,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) .remove = aerdrv_unload,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) .suspend = aerdrv_suspend,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) .resume = aerdrv_resume,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) };
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) Below is a sample code for registering/unregistering a service
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) driver.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) static int __init aerdrv_service_init(void)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) int retval = 0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) retval = pcie_port_service_register(&root_aerdrv);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) if (!retval) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) /*
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) * FIX ME
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) return retval;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) static void __exit aerdrv_service_exit(void)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) pcie_port_service_unregister(&root_aerdrv);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) module_init(aerdrv_service_init);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) module_exit(aerdrv_service_exit);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) Possible Resource Conflicts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) ===========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) Since all service drivers of a PCI-PCI Bridge Port device are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) allowed to run simultaneously, below lists a few of possible resource
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) conflicts with proposed solutions.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) MSI and MSI-X Vector Resource
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) -----------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) Once MSI or MSI-X interrupts are enabled on a device, it stays in this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) mode until they are disabled again. Since service drivers of the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) PCI-PCI Bridge port share the same physical device, if an individual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) service driver enables or disables MSI/MSI-X mode it may result
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) unpredictable behavior.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) To avoid this situation all service drivers are not permitted to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) switch interrupt mode on its device. The PCI Express Port Bus driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) is responsible for determining the interrupt mode and this should be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) transparent to service drivers. Service drivers need to know only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) the vector IRQ assigned to the field irq of struct pcie_device, which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) is passed in when the PCI Express Port Bus driver probes each service
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) driver. Service drivers should use (struct pcie_device*)dev->irq to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) call request_irq/free_irq. In addition, the interrupt mode is stored
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) in the field interrupt_mode of struct pcie_device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) PCI Memory/IO Mapped Regions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) ----------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) Service drivers for PCI Express Power Management (PME), Advanced
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) Error Reporting (AER), Hot-Plug (HP) and Virtual Channel (VC) access
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) PCI configuration space on the PCI Express port. In all cases the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) registers accessed are independent of each other. This patch assumes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) that all service drivers will be well behaved and not overwrite
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210) other service driver's configuration settings.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) PCI Config Registers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) --------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) Each service driver runs its PCI config operations on its own
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) capability structure except the PCI Express capability structure, in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) which Root Control register and Device Control register are shared
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) between PME and AER. This patch assumes that all service drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) will be well behaved and not overwrite other service driver's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) configuration settings.