^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) ============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) Subsystem Trace Points: kmem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) ============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) The kmem tracing system captures events related to object and page allocation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) within the kernel. Broadly speaking there are five major subheadings.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) - Slab allocation of small objects of unknown type (kmalloc)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) - Slab allocation of small objects of known type
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) - Page allocation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) - Per-CPU Allocator Activity
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) - External Fragmentation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) This document describes what each of the tracepoints is and why they
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) might be useful.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) 1. Slab allocation of small objects of unknown type
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) ===================================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) kmalloc call_site=%lx ptr=%p bytes_req=%zu bytes_alloc=%zu gfp_flags=%s
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) kmalloc_node call_site=%lx ptr=%p bytes_req=%zu bytes_alloc=%zu gfp_flags=%s node=%d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) kfree call_site=%lx ptr=%p
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) Heavy activity for these events may indicate that a specific cache is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) justified, particularly if kmalloc slab pages are getting significantly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) internal fragmented as a result of the allocation pattern. By correlating
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) kmalloc with kfree, it may be possible to identify memory leaks and where
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) the allocation sites were.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) 2. Slab allocation of small objects of known type
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) =================================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) kmem_cache_alloc call_site=%lx ptr=%p bytes_req=%zu bytes_alloc=%zu gfp_flags=%s
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) kmem_cache_alloc_node call_site=%lx ptr=%p bytes_req=%zu bytes_alloc=%zu gfp_flags=%s node=%d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) kmem_cache_free call_site=%lx ptr=%p
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) These events are similar in usage to the kmalloc-related events except that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) it is likely easier to pin the event down to a specific cache. At the time
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) of writing, no information is available on what slab is being allocated from,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) but the call_site can usually be used to extrapolate that information.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) 3. Page allocation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46) ==================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) mm_page_alloc page=%p pfn=%lu order=%d migratetype=%d gfp_flags=%s
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) mm_page_alloc_zone_locked page=%p pfn=%lu order=%u migratetype=%d cpu=%d percpu_refill=%d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) mm_page_free page=%p pfn=%lu order=%d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) mm_page_free_batched page=%p pfn=%lu order=%d cold=%d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) These four events deal with page allocation and freeing. mm_page_alloc is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) a simple indicator of page allocator activity. Pages may be allocated from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) the per-CPU allocator (high performance) or the buddy allocator.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) If pages are allocated directly from the buddy allocator, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) mm_page_alloc_zone_locked event is triggered. This event is important as high
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) amounts of activity imply high activity on the zone->lock. Taking this lock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) impairs performance by disabling interrupts, dirtying cache lines between
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) CPUs and serialising many CPUs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) When a page is freed directly by the caller, the only mm_page_free event
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) is triggered. Significant amounts of activity here could indicate that the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) callers should be batching their activities.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) When pages are freed in batch, the also mm_page_free_batched is triggered.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) Broadly speaking, pages are taken off the LRU lock in bulk and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) freed in batch with a page list. Significant amounts of activity here could
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) indicate that the system is under memory pressure and can also indicate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) contention on the zone->lru_lock.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) 4. Per-CPU Allocator Activity
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) =============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) mm_page_alloc_zone_locked page=%p pfn=%lu order=%u migratetype=%d cpu=%d percpu_refill=%d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79) mm_page_pcpu_drain page=%p pfn=%lu order=%d cpu=%d migratetype=%d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) In front of the page allocator is a per-cpu page allocator. It exists only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) for order-0 pages, reduces contention on the zone->lock and reduces the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) amount of writing on struct page.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) When a per-CPU list is empty or pages of the wrong type are allocated,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) the zone->lock will be taken once and the per-CPU list refilled. The event
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) triggered is mm_page_alloc_zone_locked for each page allocated with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) event indicating whether it is for a percpu_refill or not.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) When the per-CPU list is too full, a number of pages are freed, each one
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) which triggers a mm_page_pcpu_drain event.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) The individual nature of the events is so that pages can be tracked
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) between allocation and freeing. A number of drain or refill pages that occur
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) consecutively imply the zone->lock being taken once. Large amounts of per-CPU
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) refills and drains could imply an imbalance between CPUs where too much work
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) is being concentrated in one place. It could also indicate that the per-CPU
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) lists should be a larger size. Finally, large amounts of refills on one CPU
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) and drains on another could be a factor in causing large amounts of cache
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) line bounces due to writes between CPUs and worth investigating if pages
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) can be allocated and freed on the same CPU through some algorithm change.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) 5. External Fragmentation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) =========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) mm_page_alloc_extfrag page=%p pfn=%lu alloc_order=%d fallback_order=%d pageblock_order=%d alloc_migratetype=%d fallback_migratetype=%d fragmenting=%d change_ownership=%d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) External fragmentation affects whether a high-order allocation will be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) successful or not. For some types of hardware, this is important although
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) it is avoided where possible. If the system is using huge pages and needs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) to be able to resize the pool over the lifetime of the system, this value
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) is important.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) Large numbers of this event implies that memory is fragmenting and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) high-order allocations will start failing at some time in the future. One
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) means of reducing the occurrence of this event is to increase the size of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) min_free_kbytes in increments of 3*pageblock_size*nr_online_nodes where
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) pageblock_size is usually the size of the default hugepage size.
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags