Orange Pi5 kernel

^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   1) .. _admin_guide_ksm:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   2) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   3) =======================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   4) Kernel Samepage Merging
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   5) =======================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   6) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   7) Overview
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   8) ========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   9) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  10) KSM is a memory-saving de-duplication feature, enabled by CONFIG_KSM=y,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  11) added to the Linux kernel in 2.6.32.  See ``mm/ksm.c`` for its implementation,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  12) and http://lwn.net/Articles/306704/ and https://lwn.net/Articles/330589/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  13) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  14) KSM was originally developed for use with KVM (where it was known as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  15) Kernel Shared Memory), to fit more virtual machines into physical memory,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  16) by sharing the data common between them.  But it can be useful to any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  17) application which generates many instances of the same data.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  18) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  19) The KSM daemon ksmd periodically scans those areas of user memory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  20) which have been registered with it, looking for pages of identical
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  21) content which can be replaced by a single write-protected page (which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  22) is automatically copied if a process later wants to update its
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  23) content). The amount of pages that KSM daemon scans in a single pass
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  24) and the time between the passes are configured using :ref:`sysfs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  25) intraface <ksm_sysfs>`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  26) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  27) KSM only merges anonymous (private) pages, never pagecache (file) pages.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  28) KSM's merged pages were originally locked into kernel memory, but can now
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  29) be swapped out just like other user pages (but sharing is broken when they
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  30) are swapped back in: ksmd must rediscover their identity and merge again).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  31) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  32) Controlling KSM with madvise
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  33) ============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  34) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  35) KSM only operates on those areas of address space which an application
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  36) has advised to be likely candidates for merging, by using the madvise(2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  37) system call::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  38) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  39) 	int madvise(addr, length, MADV_MERGEABLE)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  40) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  41) The app may call
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  42) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  43) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  44) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  45) 	int madvise(addr, length, MADV_UNMERGEABLE)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  46) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  47) to cancel that advice and restore unshared pages: whereupon KSM
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  48) unmerges whatever it merged in that range.  Note: this unmerging call
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  49) may suddenly require more memory than is available - possibly failing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  50) with EAGAIN, but more probably arousing the Out-Of-Memory killer.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  51) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  52) If KSM is not configured into the running kernel, madvise MADV_MERGEABLE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  53) and MADV_UNMERGEABLE simply fail with EINVAL.  If the running kernel was
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  54) built with CONFIG_KSM=y, those calls will normally succeed: even if the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  55) KSM daemon is not currently running, MADV_MERGEABLE still registers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  56) the range for whenever the KSM daemon is started; even if the range
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  57) cannot contain any pages which KSM could actually merge; even if
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  58) MADV_UNMERGEABLE is applied to a range which was never MADV_MERGEABLE.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  59) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  60) If a region of memory must be split into at least one new MADV_MERGEABLE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  61) or MADV_UNMERGEABLE region, the madvise may return ENOMEM if the process
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  62) will exceed ``vm.max_map_count`` (see Documentation/admin-guide/sysctl/vm.rst).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  63) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  64) Like other madvise calls, they are intended for use on mapped areas of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  65) the user address space: they will report ENOMEM if the specified range
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  66) includes unmapped gaps (though working on the intervening mapped areas),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  67) and might fail with EAGAIN if not enough memory for internal structures.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  68) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  69) Applications should be considerate in their use of MADV_MERGEABLE,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  70) restricting its use to areas likely to benefit.  KSM's scans may use a lot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  71) of processing power: some installations will disable KSM for that reason.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  72) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  73) .. _ksm_sysfs:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  74) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  75) KSM daemon sysfs interface
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  76) ==========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  77) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  78) The KSM daemon is controlled by sysfs files in ``/sys/kernel/mm/ksm/``,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  79) readable by all but writable only by root:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  80) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  81) pages_to_scan
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  82)         how many pages to scan before ksmd goes to sleep
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  83)         e.g. ``echo 100 > /sys/kernel/mm/ksm/pages_to_scan``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  84) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  85)         Default: 100 (chosen for demonstration purposes)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  86) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  87) sleep_millisecs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  88)         how many milliseconds ksmd should sleep before next scan
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  89)         e.g. ``echo 20 > /sys/kernel/mm/ksm/sleep_millisecs``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  90) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  91)         Default: 20 (chosen for demonstration purposes)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  92) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  93) merge_across_nodes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  94)         specifies if pages from different NUMA nodes can be merged.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  95)         When set to 0, ksm merges only pages which physically reside
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  96)         in the memory area of same NUMA node. That brings lower
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  97)         latency to access of shared pages. Systems with more nodes, at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  98)         significant NUMA distances, are likely to benefit from the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  99)         lower latency of setting 0. Smaller systems, which need to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100)         minimize memory usage, are likely to benefit from the greater
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101)         sharing of setting 1 (default). You may wish to compare how
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)         your system performs under each setting, before deciding on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103)         which to use. ``merge_across_nodes`` setting can be changed only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104)         when there are no ksm shared pages in the system: set run 2 to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105)         unmerge pages first, then to 1 after changing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106)         ``merge_across_nodes``, to remerge according to the new setting.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)         Default: 1 (merging across nodes as in earlier releases)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) run
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111)         * set to 0 to stop ksmd from running but keep merged pages,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112)         * set to 1 to run ksmd e.g. ``echo 1 > /sys/kernel/mm/ksm/run``,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113)         * set to 2 to stop ksmd and unmerge all pages currently merged, but
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) 	  leave mergeable areas registered for next run.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116)         Default: 0 (must be changed to 1 to activate KSM, except if
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117)         CONFIG_SYSFS is disabled)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) use_zero_pages
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)         specifies whether empty pages (i.e. allocated pages that only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121)         contain zeroes) should be treated specially.  When set to 1,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122)         empty pages are merged with the kernel zero page(s) instead of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123)         with each other as it would happen normally. This can improve
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124)         the performance on architectures with coloured zero pages,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125)         depending on the workload. Care should be taken when enabling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126)         this setting, as it can potentially degrade the performance of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127)         KSM for some workloads, for example if the checksums of pages
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128)         candidate for merging match the checksum of an empty
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)         page. This setting can be changed at any time, it is only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130)         effective for pages merged after the change.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132)         Default: 0 (normal KSM behaviour as in earlier releases)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) max_page_sharing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135)         Maximum sharing allowed for each KSM page. This enforces a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136)         deduplication limit to avoid high latency for virtual memory
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137)         operations that involve traversal of the virtual mappings that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)         share the KSM page. The minimum value is 2 as a newly created
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)         KSM page will have at least two sharers. The higher this value
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140)         the faster KSM will merge the memory and the higher the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)         deduplication factor will be, but the slower the worst case
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142)         virtual mappings traversal could be for any given KSM
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143)         page. Slowing down this traversal means there will be higher
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144)         latency for certain virtual memory operations happening during
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145)         swapping, compaction, NUMA balancing and page migration, in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146)         turn decreasing responsiveness for the caller of those virtual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)         memory operations. The scheduler latency of other tasks not
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148)         involved with the VM operations doing the virtual mappings
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149)         traversal is not affected by this parameter as these
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150)         traversals are always schedule friendly themselves.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) stable_node_chains_prune_millisecs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153)         specifies how frequently KSM checks the metadata of the pages
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154)         that hit the deduplication limit for stale information.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155)         Smaller milllisecs values will free up the KSM metadata with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156)         lower latency, but they will make ksmd use more CPU during the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157)         scan. It's a noop if not a single KSM page hit the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158)         ``max_page_sharing`` yet.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) The effectiveness of KSM and MADV_MERGEABLE is shown in ``/sys/kernel/mm/ksm/``:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) pages_shared
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163)         how many shared pages are being used
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) pages_sharing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)         how many more sites are sharing them i.e. how much saved
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) pages_unshared
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)         how many pages unique but repeatedly checked for merging
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) pages_volatile
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)         how many pages changing too fast to be placed in a tree
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) full_scans
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171)         how many times all mergeable areas have been scanned
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) stable_node_chains
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173)         the number of KSM pages that hit the ``max_page_sharing`` limit
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) stable_node_dups
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175)         number of duplicated KSM pages
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) A high ratio of ``pages_sharing`` to ``pages_shared`` indicates good
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) sharing, but a high ratio of ``pages_unshared`` to ``pages_sharing``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) indicates wasted effort.  ``pages_volatile`` embraces several
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) different kinds of activity, but a high proportion there would also
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) indicate poor use of madvise MADV_MERGEABLE.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) The maximum possible ``pages_sharing/pages_shared`` ratio is limited by the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) ``max_page_sharing`` tunable. To increase the ratio ``max_page_sharing`` must
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) be increased accordingly.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) --
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) Izik Eidus,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) Hugh Dickins, 17 Nov 2009