diff options
Diffstat (limited to 'Documentation/admin-guide/cgroup-v1')
| -rw-r--r-- | Documentation/admin-guide/cgroup-v1/blkio-controller.rst | 155 | ||||
| -rw-r--r-- | Documentation/admin-guide/cgroup-v1/cgroups.rst | 2 | ||||
| -rw-r--r-- | Documentation/admin-guide/cgroup-v1/cpusets.rst | 17 | ||||
| -rw-r--r-- | Documentation/admin-guide/cgroup-v1/hugetlb.rst | 111 | ||||
| -rw-r--r-- | Documentation/admin-guide/cgroup-v1/index.rst | 3 | ||||
| -rw-r--r-- | Documentation/admin-guide/cgroup-v1/memcg_test.rst | 27 | ||||
| -rw-r--r-- | Documentation/admin-guide/cgroup-v1/memory.rst | 420 | ||||
| -rw-r--r-- | Documentation/admin-guide/cgroup-v1/misc.rst | 4 | ||||
| -rw-r--r-- | Documentation/admin-guide/cgroup-v1/rdma.rst | 2 |
9 files changed, 428 insertions, 313 deletions
diff --git a/Documentation/admin-guide/cgroup-v1/blkio-controller.rst b/Documentation/admin-guide/cgroup-v1/blkio-controller.rst index 36d43ae7dc13..dabb80cdd25a 100644 --- a/Documentation/admin-guide/cgroup-v1/blkio-controller.rst +++ b/Documentation/admin-guide/cgroup-v1/blkio-controller.rst @@ -17,36 +17,37 @@ level logical devices like device mapper. HOWTO ===== + Throttling/Upper Limit policy ----------------------------- -- Enable Block IO controller:: +Enable Block IO controller:: CONFIG_BLK_CGROUP=y -- Enable throttling in block layer:: +Enable throttling in block layer:: CONFIG_BLK_DEV_THROTTLING=y -- Mount blkio controller (see cgroups.txt, Why are cgroups needed?):: +Mount blkio controller (see cgroups.txt, Why are cgroups needed?):: mount -t cgroup -o blkio none /sys/fs/cgroup/blkio -- Specify a bandwidth rate on particular device for root group. The format - for policy is "<major>:<minor> <bytes_per_second>":: +Specify a bandwidth rate on particular device for root group. The format +for policy is "<major>:<minor> <bytes_per_second>":: echo "8:16 1048576" > /sys/fs/cgroup/blkio/blkio.throttle.read_bps_device - Above will put a limit of 1MB/second on reads happening for root group - on device having major/minor number 8:16. +This will put a limit of 1MB/second on reads happening for root group +on device having major/minor number 8:16. -- Run dd to read a file and see if rate is throttled to 1MB/s or not:: +Run dd to read a file and see if rate is throttled to 1MB/s or not:: # dd iflag=direct if=/mnt/common/zerofile of=/dev/null bs=4K count=1024 1024+0 records in 1024+0 records out 4194304 bytes (4.2 MB) copied, 4.0001 s, 1.0 MB/s - Limits for writes can be put using blkio.throttle.write_bps_device file. +Limits for writes can be put using blkio.throttle.write_bps_device file. Hierarchical Cgroups ==================== @@ -79,85 +80,89 @@ following:: Various user visible config options =================================== -CONFIG_BLK_CGROUP - - Block IO controller. -CONFIG_BFQ_CGROUP_DEBUG - - Debug help. Right now some additional stats file show up in cgroup + CONFIG_BLK_CGROUP + Block IO controller. + + CONFIG_BFQ_CGROUP_DEBUG + Debug help. Right now some additional stats file show up in cgroup if this option is enabled. -CONFIG_BLK_DEV_THROTTLING - - Enable block device throttling support in block layer. + CONFIG_BLK_DEV_THROTTLING + Enable block device throttling support in block layer. Details of cgroup files ======================= + Proportional weight policy files -------------------------------- -- blkio.weight - - Specifies per cgroup weight. This is default weight of the group - on all the devices until and unless overridden by per device rule. - (See blkio.weight_device). - Currently allowed range of weights is from 10 to 1000. -- blkio.weight_device - - One can specify per cgroup per device rules using this interface. - These rules override the default value of group weight as specified - by blkio.weight. + blkio.bfq.weight + Specifies per cgroup weight. This is default weight of the group + on all the devices until and unless overridden by per device rule + (see `blkio.bfq.weight_device` below). + + Currently allowed range of weights is from 1 to 1000. For more details, + see Documentation/block/bfq-iosched.rst. + + blkio.bfq.weight_device + Specifies per cgroup per device weights, overriding the default group + weight. For more details, see Documentation/block/bfq-iosched.rst. Following is the format:: - # echo dev_maj:dev_minor weight > blkio.weight_device + # echo dev_maj:dev_minor weight > blkio.bfq.weight_device Configure weight=300 on /dev/sdb (8:16) in this cgroup:: - # echo 8:16 300 > blkio.weight_device - # cat blkio.weight_device + # echo 8:16 300 > blkio.bfq.weight_device + # cat blkio.bfq.weight_device dev weight 8:16 300 Configure weight=500 on /dev/sda (8:0) in this cgroup:: - # echo 8:0 500 > blkio.weight_device - # cat blkio.weight_device + # echo 8:0 500 > blkio.bfq.weight_device + # cat blkio.bfq.weight_device dev weight 8:0 500 8:16 300 Remove specific weight for /dev/sda in this cgroup:: - # echo 8:0 0 > blkio.weight_device - # cat blkio.weight_device + # echo 8:0 0 > blkio.bfq.weight_device + # cat blkio.bfq.weight_device dev weight 8:16 300 -- blkio.time - - disk time allocated to cgroup per device in milliseconds. First + blkio.time + Disk time allocated to cgroup per device in milliseconds. First two fields specify the major and minor number of the device and third field specifies the disk time allocated to group in milliseconds. -- blkio.sectors - - number of sectors transferred to/from disk by the group. First + blkio.sectors + Number of sectors transferred to/from disk by the group. First two fields specify the major and minor number of the device and third field specifies the number of sectors transferred by the group to/from the device. -- blkio.io_service_bytes - - Number of bytes transferred to/from the disk by the group. These + blkio.io_service_bytes + Number of bytes transferred to/from the disk by the group. These are further divided by the type of operation - read or write, sync or async. First two fields specify the major and minor number of the device, third field specifies the operation type and the fourth field specifies the number of bytes. -- blkio.io_serviced - - Number of IOs (bio) issued to the disk by the group. These + blkio.io_serviced + Number of IOs (bio) issued to the disk by the group. These are further divided by the type of operation - read or write, sync or async. First two fields specify the major and minor number of the device, third field specifies the operation type and the fourth field specifies the number of IOs. -- blkio.io_service_time - - Total amount of time between request dispatch and request completion + blkio.io_service_time + Total amount of time between request dispatch and request completion for the IOs done by this cgroup. This is in nanoseconds to make it meaningful for flash devices too. For devices with queue depth of 1, this time represents the actual service time. When queue_depth > 1, @@ -170,8 +175,8 @@ Proportional weight policy files specifies the operation type and the fourth field specifies the io_service_time in ns. -- blkio.io_wait_time - - Total amount of time the IOs for this cgroup spent waiting in the + blkio.io_wait_time + Total amount of time the IOs for this cgroup spent waiting in the scheduler queues for service. This can be greater than the total time elapsed since it is cumulative io_wait_time for all IOs. It is not a measure of total time the cgroup spent waiting but rather a measure of @@ -185,24 +190,24 @@ Proportional weight policy files minor number of the device, third field specifies the operation type and the fourth field specifies the io_wait_time in ns. -- blkio.io_merged - - Total number of bios/requests merged into requests belonging to this + blkio.io_merged + Total number of bios/requests merged into requests belonging to this cgroup. This is further divided by the type of operation - read or write, sync or async. -- blkio.io_queued - - Total number of requests queued up at any given instant for this + blkio.io_queued + Total number of requests queued up at any given instant for this cgroup. This is further divided by the type of operation - read or write, sync or async. -- blkio.avg_queue_size - - Debugging aid only enabled if CONFIG_BFQ_CGROUP_DEBUG=y. + blkio.avg_queue_size + Debugging aid only enabled if CONFIG_BFQ_CGROUP_DEBUG=y. The average queue size for this cgroup over the entire time of this cgroup's existence. Queue size samples are taken each time one of the queues of this cgroup gets a timeslice. -- blkio.group_wait_time - - Debugging aid only enabled if CONFIG_BFQ_CGROUP_DEBUG=y. + blkio.group_wait_time + Debugging aid only enabled if CONFIG_BFQ_CGROUP_DEBUG=y. This is the amount of time the cgroup had to wait since it became busy (i.e., went from 0 to 1 request queued) to get a timeslice for one of its queues. This is different from the io_wait_time which is the @@ -212,8 +217,8 @@ Proportional weight policy files will only report the group_wait_time accumulated till the last time it got a timeslice and will not include the current delta. -- blkio.empty_time - - Debugging aid only enabled if CONFIG_BFQ_CGROUP_DEBUG=y. + blkio.empty_time + Debugging aid only enabled if CONFIG_BFQ_CGROUP_DEBUG=y. This is the amount of time a cgroup spends without any pending requests when not being served, i.e., it does not include any time spent idling for one of the queues of the cgroup. This is in @@ -221,8 +226,8 @@ Proportional weight policy files the stat will only report the empty_time accumulated till the last time it had a pending request and will not include the current delta. -- blkio.idle_time - - Debugging aid only enabled if CONFIG_BFQ_CGROUP_DEBUG=y. + blkio.idle_time + Debugging aid only enabled if CONFIG_BFQ_CGROUP_DEBUG=y. This is the amount of time spent by the IO scheduler idling for a given cgroup in anticipation of a better request than the existing ones from other queues/cgroups. This is in nanoseconds. If this is read @@ -230,60 +235,60 @@ Proportional weight policy files idle_time accumulated till the last idle period and will not include the current delta. -- blkio.dequeue - - Debugging aid only enabled if CONFIG_BFQ_CGROUP_DEBUG=y. This + blkio.dequeue + Debugging aid only enabled if CONFIG_BFQ_CGROUP_DEBUG=y. This gives the statistics about how many a times a group was dequeued from service tree of the device. First two fields specify the major and minor number of the device and third field specifies the number of times a group was dequeued from a particular device. -- blkio.*_recursive - - Recursive version of various stats. These files show the + blkio.*_recursive + Recursive version of various stats. These files show the same information as their non-recursive counterparts but include stats from all the descendant cgroups. Throttling/Upper limit policy files ----------------------------------- -- blkio.throttle.read_bps_device - - Specifies upper limit on READ rate from the device. IO rate is + blkio.throttle.read_bps_device + Specifies upper limit on READ rate from the device. IO rate is specified in bytes per second. Rules are per device. Following is the format:: echo "<major>:<minor> <rate_bytes_per_second>" > /cgrp/blkio.throttle.read_bps_device -- blkio.throttle.write_bps_device - - Specifies upper limit on WRITE rate to the device. IO rate is + blkio.throttle.write_bps_device + Specifies upper limit on WRITE rate to the device. IO rate is specified in bytes per second. Rules are per device. Following is the format:: echo "<major>:<minor> <rate_bytes_per_second>" > /cgrp/blkio.throttle.write_bps_device -- blkio.throttle.read_iops_device - - Specifies upper limit on READ rate from the device. IO rate is + blkio.throttle.read_iops_device + Specifies upper limit on READ rate from the device. IO rate is specified in IO per second. Rules are per device. Following is the format:: echo "<major>:<minor> <rate_io_per_second>" > /cgrp/blkio.throttle.read_iops_device -- blkio.throttle.write_iops_device - - Specifies upper limit on WRITE rate to the device. IO rate is + blkio.throttle.write_iops_device + Specifies upper limit on WRITE rate to the device. IO rate is specified in io per second. Rules are per device. Following is the format:: echo "<major>:<minor> <rate_io_per_second>" > /cgrp/blkio.throttle.write_iops_device -Note: If both BW and IOPS rules are specified for a device, then IO is - subjected to both the constraints. + Note: If both BW and IOPS rules are specified for a device, then IO is + subjected to both the constraints. -- blkio.throttle.io_serviced - - Number of IOs (bio) issued to the disk by the group. These + blkio.throttle.io_serviced + Number of IOs (bio) issued to the disk by the group. These are further divided by the type of operation - read or write, sync or async. First two fields specify the major and minor number of the device, third field specifies the operation type and the fourth field specifies the number of IOs. -- blkio.throttle.io_service_bytes - - Number of bytes transferred to/from the disk by the group. These + blkio.throttle.io_service_bytes + Number of bytes transferred to/from the disk by the group. These are further divided by the type of operation - read or write, sync or async. First two fields specify the major and minor number of the device, third field specifies the operation type and the fourth field @@ -291,6 +296,6 @@ Note: If both BW and IOPS rules are specified for a device, then IO is Common files among various policies ----------------------------------- -- blkio.reset_stats - - Writing an int to this file will result in resetting all the stats + blkio.reset_stats + Writing an int to this file will result in resetting all the stats for that cgroup. diff --git a/Documentation/admin-guide/cgroup-v1/cgroups.rst b/Documentation/admin-guide/cgroup-v1/cgroups.rst index b0688011ed06..9343148ee993 100644 --- a/Documentation/admin-guide/cgroup-v1/cgroups.rst +++ b/Documentation/admin-guide/cgroup-v1/cgroups.rst @@ -80,6 +80,8 @@ access. For example, cpusets (see Documentation/admin-guide/cgroup-v1/cpusets.rs you to associate a set of CPUs and a set of memory nodes with the tasks in each cgroup. +.. _cgroups-why-needed: + 1.2 Why are cgroups needed ? ---------------------------- diff --git a/Documentation/admin-guide/cgroup-v1/cpusets.rst b/Documentation/admin-guide/cgroup-v1/cpusets.rst index 86a6ae995d54..7d3415eea05d 100644 --- a/Documentation/admin-guide/cgroup-v1/cpusets.rst +++ b/Documentation/admin-guide/cgroup-v1/cpusets.rst @@ -1,3 +1,5 @@ +.. _cpusets: + ======= CPUSETS ======= @@ -177,7 +179,7 @@ files describing that cpuset: - cpuset.mem_hardwall flag: is memory allocation hardwalled - cpuset.memory_pressure: measure of how much paging pressure in cpuset - cpuset.memory_spread_page flag: if set, spread page cache evenly on allowed nodes - - cpuset.memory_spread_slab flag: if set, spread slab cache evenly on allowed nodes + - cpuset.memory_spread_slab flag: OBSOLETE. Doesn't have any function. - cpuset.sched_load_balance flag: if set, load balance within CPUs on that cpuset - cpuset.sched_relax_domain_level: the searching range when migrating tasks @@ -223,6 +225,17 @@ cpu_online_mask using a CPU hotplug notifier, and the mems file automatically tracks the value of node_states[N_MEMORY]--i.e., nodes with memory--using the cpuset_track_online_nodes() hook. +The cpuset.effective_cpus and cpuset.effective_mems files are +normally read-only copies of cpuset.cpus and cpuset.mems files +respectively. If the cpuset cgroup filesystem is mounted with the +special "cpuset_v2_mode" option, the behavior of these files will become +similar to the corresponding files in cpuset v2. In other words, hotplug +events will not change cpuset.cpus and cpuset.mems. Those events will +only affect cpuset.effective_cpus and cpuset.effective_mems which show +the actual cpus and memory nodes that are currently used by this cpuset. +See Documentation/admin-guide/cgroup-v2.rst for more information about +cpuset v2 behavior. + 1.4 What are exclusive cpusets ? -------------------------------- @@ -706,7 +719,7 @@ There are ways to query or modify cpusets: cat, rmdir commands from the shell, or their equivalent from C. - via the C library libcpuset. - via the C library libcgroup. - (http://sourceforge.net/projects/libcg/) + (https://github.com/libcgroup/libcgroup/) - via the python application cset. (http://code.google.com/p/cpuset/) diff --git a/Documentation/admin-guide/cgroup-v1/hugetlb.rst b/Documentation/admin-guide/cgroup-v1/hugetlb.rst index a3902aa253a9..493a8e386700 100644 --- a/Documentation/admin-guide/cgroup-v1/hugetlb.rst +++ b/Documentation/admin-guide/cgroup-v1/hugetlb.rst @@ -2,13 +2,6 @@ HugeTLB Controller ================== -The HugeTLB controller allows to limit the HugeTLB usage per control group and -enforces the controller limit during page fault. Since HugeTLB doesn't -support page reclaim, enforcing the limit at page fault time implies that, -the application will get SIGBUS signal if it tries to access HugeTLB pages -beyond its limit. This requires the application to know beforehand how much -HugeTLB pages it would require for its use. - HugeTLB controller can be created by first mounting the cgroup filesystem. # mount -t cgroup -o hugetlb none /sys/fs/cgroup @@ -28,23 +21,119 @@ process (bash) into it. Brief summary of control files:: - hugetlb.<hugepagesize>.limit_in_bytes # set/show limit of "hugepagesize" hugetlb usage - hugetlb.<hugepagesize>.max_usage_in_bytes # show max "hugepagesize" hugetlb usage recorded - hugetlb.<hugepagesize>.usage_in_bytes # show current usage for "hugepagesize" hugetlb - hugetlb.<hugepagesize>.failcnt # show the number of allocation failure due to HugeTLB limit + hugetlb.<hugepagesize>.rsvd.limit_in_bytes # set/show limit of "hugepagesize" hugetlb reservations + hugetlb.<hugepagesize>.rsvd.max_usage_in_bytes # show max "hugepagesize" hugetlb reservations and no-reserve faults + hugetlb.<hugepagesize>.rsvd.usage_in_bytes # show current reservations and no-reserve faults for "hugepagesize" hugetlb + hugetlb.<hugepagesize>.rsvd.failcnt # show the number of allocation failure due to HugeTLB reservation limit + hugetlb.<hugepagesize>.limit_in_bytes # set/show limit of "hugepagesize" hugetlb faults + hugetlb.<hugepagesize>.max_usage_in_bytes # show max "hugepagesize" hugetlb usage recorded + hugetlb.<hugepagesize>.usage_in_bytes # show current usage for "hugepagesize" hugetlb + hugetlb.<hugepagesize>.failcnt # show the number of allocation failure due to HugeTLB usage limit + hugetlb.<hugepagesize>.numa_stat # show the numa information of the hugetlb memory charged to this cgroup For a system supporting three hugepage sizes (64k, 32M and 1G), the control files include:: hugetlb.1GB.limit_in_bytes hugetlb.1GB.max_usage_in_bytes + hugetlb.1GB.numa_stat hugetlb.1GB.usage_in_bytes hugetlb.1GB.failcnt + hugetlb.1GB.rsvd.limit_in_bytes + hugetlb.1GB.rsvd.max_usage_in_bytes + hugetlb.1GB.rsvd.usage_in_bytes + hugetlb.1GB.rsvd.failcnt hugetlb.64KB.limit_in_bytes hugetlb.64KB.max_usage_in_bytes + hugetlb.64KB.numa_stat hugetlb.64KB.usage_in_bytes hugetlb.64KB.failcnt + hugetlb.64KB.rsvd.limit_in_bytes + hugetlb.64KB.rsvd.max_usage_in_bytes + hugetlb.64KB.rsvd.usage_in_bytes + hugetlb.64KB.rsvd.failcnt hugetlb.32MB.limit_in_bytes hugetlb.32MB.max_usage_in_bytes + hugetlb.32MB.numa_stat hugetlb.32MB.usage_in_bytes hugetlb.32MB.failcnt + hugetlb.32MB.rsvd.limit_in_bytes + hugetlb.32MB.rsvd.max_usage_in_bytes + hugetlb.32MB.rsvd.usage_in_bytes + hugetlb.32MB.rsvd.failcnt + + +1. Page fault accounting + +:: + + hugetlb.<hugepagesize>.limit_in_bytes + hugetlb.<hugepagesize>.max_usage_in_bytes + hugetlb.<hugepagesize>.usage_in_bytes + hugetlb.<hugepagesize>.failcnt + +The HugeTLB controller allows users to limit the HugeTLB usage (page fault) per +control group and enforces the limit during page fault. Since HugeTLB +doesn't support page reclaim, enforcing the limit at page fault time implies +that, the application will get SIGBUS signal if it tries to fault in HugeTLB +pages beyond its limit. Therefore the application needs to know exactly how many +HugeTLB pages it uses before hand, and the sysadmin needs to make sure that +there are enough available on the machine for all the users to avoid processes +getting SIGBUS. + + +2. Reservation accounting + +:: + + hugetlb.<hugepagesize>.rsvd.limit_in_bytes + hugetlb.<hugepagesize>.rsvd.max_usage_in_bytes + hugetlb.<hugepagesize>.rsvd.usage_in_bytes + hugetlb.<hugepagesize>.rsvd.failcnt + +The HugeTLB controller allows to limit the HugeTLB reservations per control +group and enforces the controller limit at reservation time and at the fault of +HugeTLB memory for which no reservation exists. Since reservation limits are +enforced at reservation time (on mmap or shget), reservation limits never causes +the application to get SIGBUS signal if the memory was reserved before hand. For +MAP_NORESERVE allocations, the reservation limit behaves the same as the fault +limit, enforcing memory usage at fault time and causing the application to +receive a SIGBUS if it's crossing its limit. + +Reservation limits are superior to page fault limits described above, since +reservation limits are enforced at reservation time (on mmap or shget), and +never causes the application to get SIGBUS signal if the memory was reserved +before hand. This allows for easier fallback to alternatives such as +non-HugeTLB memory for example. In the case of page fault accounting, it's very +hard to avoid processes getting SIGBUS since the sysadmin needs precisely know +the HugeTLB usage of all the tasks in the system and make sure there is enough +pages to satisfy all requests. Avoiding tasks getting SIGBUS on overcommited +systems is practically impossible with page fault accounting. + + +3. Caveats with shared memory + +For shared HugeTLB memory, both HugeTLB reservation and page faults are charged +to the first task that causes the memory to be reserved or faulted, and all +subsequent uses of this reserved or faulted memory is done without charging. + +Shared HugeTLB memory is only uncharged when it is unreserved or deallocated. +This is usually when the HugeTLB file is deleted, and not when the task that +caused the reservation or fault has exited. + + +4. Caveats with HugeTLB cgroup offline. + +When a HugeTLB cgroup goes offline with some reservations or faults still +charged to it, the behavior is as follows: + +- The fault charges are charged to the parent HugeTLB cgroup (reparented), +- the reservation charges remain on the offline HugeTLB cgroup. + +This means that if a HugeTLB cgroup gets offlined while there is still HugeTLB +reservations charged to it, that cgroup persists as a zombie until all HugeTLB +reservations are uncharged. HugeTLB reservations behave in this manner to match +the memory controller whose cgroups also persist as zombie until all charged +memory is uncharged. Also, the tracking of HugeTLB reservations is a bit more +complex compared to the tracking of HugeTLB faults, so it is significantly +harder to reparent reservations at offline time. diff --git a/Documentation/admin-guide/cgroup-v1/index.rst b/Documentation/admin-guide/cgroup-v1/index.rst index 10bf48bae0b0..99fbc8a64ba9 100644 --- a/Documentation/admin-guide/cgroup-v1/index.rst +++ b/Documentation/admin-guide/cgroup-v1/index.rst @@ -1,3 +1,5 @@ +.. _cgroup-v1: + ======================== Control Groups version 1 ======================== @@ -15,6 +17,7 @@ Control Groups version 1 hugetlb memcg_test memory + misc net_cls net_prio pids diff --git a/Documentation/admin-guide/cgroup-v1/memcg_test.rst b/Documentation/admin-guide/cgroup-v1/memcg_test.rst index 3f7115e07b5d..1f128458ddea 100644 --- a/Documentation/admin-guide/cgroup-v1/memcg_test.rst +++ b/Documentation/admin-guide/cgroup-v1/memcg_test.rst @@ -62,7 +62,7 @@ Please note that implementation details can be changed. At cancel(), simply usage -= PAGE_SIZE. -Under below explanation, we assume CONFIG_MEM_RES_CTRL_SWAP=y. +Under below explanation, we assume CONFIG_SWAP=y. 4. Anonymous ============ @@ -97,7 +97,7 @@ Under below explanation, we assume CONFIG_MEM_RES_CTRL_SWAP=y. ============= Page Cache is charged at - - add_to_page_cache_locked(). + - filemap_add_folio(). The logic is very clear. (About migration, see below) @@ -133,18 +133,9 @@ Under below explanation, we assume CONFIG_MEM_RES_CTRL_SWAP=y. 8. LRU ====== - Each memcg has its own private LRU. Now, its handling is under global - VM's control (means that it's handled under global pgdat->lru_lock). - Almost all routines around memcg's LRU is called by global LRU's - list management functions under pgdat->lru_lock. - - A special function is mem_cgroup_isolate_pages(). This scans - memcg's private LRU and call __isolate_lru_page() to extract a page - from LRU. - - (By __isolate_lru_page(), the page is removed from both of global and - private LRU.) - + Each memcg has its own vector of LRUs (inactive anon, active anon, + inactive file, active file, unevictable) of pages from each node, + each LRU handled under a single lru_lock for that memcg and node. 9. Typical Tests. ================= @@ -219,13 +210,11 @@ Under below explanation, we assume CONFIG_MEM_RES_CTRL_SWAP=y. This is an easy way to test page migration, too. -9.5 mkdir/rmdir ---------------- +9.5 nested cgroups +------------------ - When using hierarchy, mkdir/rmdir test should be done. - Use tests like the following:: + Use tests like the following for testing nested cgroups:: - echo 1 >/opt/cgroup/01/memory/use_hierarchy mkdir /opt/cgroup/01/child_a mkdir /opt/cgroup/01/child_b diff --git a/Documentation/admin-guide/cgroup-v1/memory.rst b/Documentation/admin-guide/cgroup-v1/memory.rst index 0ae4f564c2d6..ca7d9402f6be 100644 --- a/Documentation/admin-guide/cgroup-v1/memory.rst +++ b/Documentation/admin-guide/cgroup-v1/memory.rst @@ -2,18 +2,18 @@ Memory Resource Controller ========================== -NOTE: +.. caution:: This document is hopelessly outdated and it asks for a complete rewrite. It still contains a useful information so we are keeping it here but make sure to check the current code if you need a deeper understanding. -NOTE: +.. note:: The Memory Resource Controller has generically been referred to as the memory controller in this document. Do not confuse memory controller used here with the memory controller that is used in hardware. -(For editors) In this document: +.. hint:: When we mention a cgroup (cgroupfs's directory) with memory controller, we call it "memory cgroup". When you see git-log and source code, you'll see patch's title and function names tend to use "memcg". @@ -23,7 +23,7 @@ Benefits and Purpose of the memory controller ============================================= The memory controller isolates the memory behaviour of a group of tasks -from the rest of the system. The article on LWN [12] mentions some probable +from the rest of the system. The article on LWN [12]_ mentions some probable uses of the memory controller. The memory controller can be used to a. Isolate an application or a group of applications @@ -55,7 +55,8 @@ Features: - Root cgroup has no limit controls. Kernel memory support is a work in progress, and the current version provides - basically functionality. (See Section 2.7) + basically functionality. (See :ref:`section 2.7 + <cgroup-v1-memory-kernel-extension>`) Brief summary of control files. @@ -64,6 +65,7 @@ Brief summary of control files. threads cgroup.procs show list of processes cgroup.event_control an interface for event_fd() + This knob is not available on CONFIG_PREEMPT_RT systems. memory.usage_in_bytes show current usage for memory (See 5.5 for details) memory.memsw.usage_in_bytes show current usage for memory+Swap @@ -75,20 +77,28 @@ Brief summary of control files. memory.max_usage_in_bytes show max memory usage recorded memory.memsw.max_usage_in_bytes show max memory+Swap usage recorded memory.soft_limit_in_bytes set/show soft limit of memory usage + This knob is not available on CONFIG_PREEMPT_RT systems. memory.stat show various statistics memory.use_hierarchy set/show hierarchical account enabled + This knob is deprecated and shouldn't be + used. memory.force_empty trigger forced page reclaim memory.pressure_level set memory pressure notifications memory.swappiness set/show swappiness parameter of vmscan (See sysctl's vm.swappiness) memory.move_charge_at_immigrate set/show controls of moving charges + This knob is deprecated and shouldn't be + used. memory.oom_control set/show oom controls. memory.numa_stat show the number of memory usage per numa node - memory.kmem.limit_in_bytes set/show hard limit for kernel memory - This knob is deprecated and shouldn't be - used. It is planned that this be removed in - the foreseeable future. + memory.kmem.limit_in_bytes Deprecated knob to set and read the kernel + memory hard limit. Kernel hard limit is not + supported since 5.16. Writing any value to + do file will not have any effect same as if + nokmem kernel parameter was specified. + Kernel memory is still charged and reported + by memory.kmem.usage_in_bytes. memory.kmem.usage_in_bytes show current kernel memory allocation memory.kmem.failcnt show the number of kernel memory usage hits limits @@ -105,16 +115,16 @@ Brief summary of control files. ========== The memory controller has a long history. A request for comments for the memory -controller was posted by Balbir Singh [1]. At the time the RFC was posted +controller was posted by Balbir Singh [1]_. At the time the RFC was posted there were several implementations for memory control. The goal of the RFC was to build consensus and agreement for the minimal features required -for memory control. The first RSS controller was posted by Balbir Singh[2] -in Feb 2007. Pavel Emelianov [3][4][5] has since posted three versions of the -RSS controller. At OLS, at the resource management BoF, everyone suggested -that we handle both page cache and RSS together. Another request was raised -to allow user space handling of OOM. The current memory controller is +for memory control. The first RSS controller was posted by Balbir Singh [2]_ +in Feb 2007. Pavel Emelianov [3]_ [4]_ [5]_ has since posted three versions +of the RSS controller. At OLS, at the resource management BoF, everyone +suggested that we handle both page cache and RSS together. Another request was +raised to allow user space handling of OOM. The current memory controller is at version 6; it combines both mapped (RSS) and unmapped Page -Cache Control [11]. +Cache Control [11]_. 2. Memory Control ================= @@ -145,7 +155,8 @@ specific data structure (mem_cgroup) associated with it. 2.2. Accounting --------------- -:: +.. code-block:: + :caption: Figure 1: Hierarchy of Accounting +--------------------+ | mem_cgroup | @@ -165,7 +176,6 @@ specific data structure (mem_cgroup) associated with it. | | | | +---------------+ +---------------+ - (Figure 1: Hierarchy of Accounting) Figure 1 shows the important aspects of the controller @@ -192,18 +202,18 @@ are not accounted. We just account pages under usual VM management. RSS pages are accounted at page_fault unless they've already been accounted for earlier. A file page will be accounted for as Page Cache when it's -inserted into inode (radix-tree). While it's mapped into the page tables of +inserted into inode (xarray). While it's mapped into the page tables of processes, duplicate accounting is carefully avoided. An RSS page is unaccounted when it's fully unmapped. A PageCache page is -unaccounted when it's removed from radix-tree. Even if RSS pages are fully +unaccounted when it's removed from xarray. Even if RSS pages are fully unmapped (by kswapd), they may exist as SwapCache in the system until they are really freed. Such SwapCaches are also accounted. -A swapped-in page is not accounted until it's mapped. +A swapped-in page is accounted after adding into swapcache. Note: The kernel does swapin-readahead and reads multiple swaps at once. -This means swapped-in pages may contain pages for other tasks than a task -causing page fault. So, we avoid accounting at swap-in I/O. +Since page's memcg recorded into swap whatever memsw enabled, the page will +be accounted after swapin. At page migration, accounting information is kept. @@ -219,21 +229,17 @@ behind this approach is that a cgroup that aggressively uses a shared page will eventually get charged for it (once it is uncharged from the cgroup that brought it in -- this will happen on memory pressure). -But see section 8.2: when moving a task to another cgroup, its pages may -be recharged to the new cgroup, if move_charge_at_immigrate has been chosen. - -Exception: If CONFIG_MEMCG_SWAP is not used. -When you do swapoff and make swapped-out pages of shmem(tmpfs) to -be backed into memory in force, charges for pages are accounted against the -caller of swapoff rather than the users of shmem. +But see :ref:`section 8.2 <cgroup-v1-memory-movable-charges>` when moving a +task to another cgroup, its pages may be recharged to the new cgroup, if +move_charge_at_immigrate has been chosen. -2.4 Swap Extension (CONFIG_MEMCG_SWAP) +2.4 Swap Extension -------------------------------------- -Swap Extension allows you to record charge for swap. A swapped-in page is -charged back to original page allocator if possible. +Swap usage is always recorded for each of cgroup. Swap Extension allows you to +read and limit it. -When swap is accounted, following files are added. +When CONFIG_SWAP is enabled, following files are added. - memory.memsw.usage_in_bytes. - memory.memsw.limit_in_bytes. @@ -247,7 +253,8 @@ In this case, setting memsw.limit_in_bytes=3G will prevent bad use of swap. By using the memsw limit, you can avoid system OOM which can be caused by swap shortage. -**why 'memory+swap' rather than swap** +2.4.1 why 'memory+swap' rather than swap +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The global LRU(kswapd) can swap out arbitrary pages. Swap-out means to move account from memory to swap...there is no change in usage of @@ -255,7 +262,8 @@ memory+swap. In other words, when we want to limit the usage of swap without affecting global LRU, memory+swap limit is better than just limiting swap from an OS point of view. -**What happens when a cgroup hits memory.memsw.limit_in_bytes** +2.4.2. What happens when a cgroup hits memory.memsw.limit_in_bytes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When a cgroup hits memory.memsw.limit_in_bytes, it's useless to do swap-out in this cgroup. Then, swap-out will not be done by cgroup routine and file @@ -271,41 +279,40 @@ global VM. When a cgroup goes over its limit, we first try to reclaim memory from the cgroup so as to make space for the new pages that the cgroup has touched. If the reclaim is unsuccessful, an OOM routine is invoked to select and kill the bulkiest task in the -cgroup. (See 10. OOM Control below.) +cgroup. (See :ref:`10. OOM Control <cgroup-v1-memory-oom-control>` below.) The reclaim algorithm has not been modified for cgroups, except that pages that are selected for reclaiming come from the per-cgroup LRU list. -NOTE: - Reclaim does not work for the root cgroup, since we cannot set any - limits on the root cgroup. +.. note:: + Reclaim does not work for the root cgroup, since we cannot set any + limits on the root cgroup. -Note2: - When panic_on_oom is set to "2", the whole system will panic. +.. note:: + When panic_on_oom is set to "2", the whole system will panic. When oom event notifier is registered, event will be delivered. -(See oom_control section) +(See :ref:`oom_control <cgroup-v1-memory-oom-control>` section) 2.6 Locking ----------- - lock_page_cgroup()/unlock_page_cgroup() should not be called under - the i_pages lock. +Lock order is as follows:: - Other lock order is following: + Page lock (PG_locked bit of page->flags) + mm->page_table_lock or split pte_lock + folio_memcg_lock (memcg->move_lock) + mapping->i_pages lock + lruvec->lru_lock. - PG_locked. - mm->page_table_lock - pgdat->lru_lock - lock_page_cgroup. +Per-node-per-memcgroup LRU (cgroup's private LRU) is guarded by +lruvec->lru_lock; PG_lru bit of page->flags is cleared before +isolating a page from its LRU under lruvec->lru_lock. - In many cases, just lock_page_cgroup() is called. +.. _cgroup-v1-memory-kernel-extension: - per-zone-per-cgroup LRU (cgroup's private LRU) is just guarded by - pgdat->lru_lock, it has no lock of its own. - -2.7 Kernel Memory Extension (CONFIG_MEMCG_KMEM) +2.7 Kernel Memory Extension ----------------------------------------------- With the Kernel memory extension, the Memory Controller is able to limit @@ -366,17 +373,17 @@ U != 0, K = unlimited: U != 0, K < U: Kernel memory is a subset of the user memory. This setup is useful in - deployments where the total amount of memory per-cgroup is overcommited. - Overcommiting kernel memory limits is definitely not recommended, since the + deployments where the total amount of memory per-cgroup is overcommitted. + Overcommitting kernel memory limits is definitely not recommended, since the box can still run out of non-reclaimable memory. In this case, the admin could set up K so that the sum of all groups is never greater than the total memory, and freely set U at the cost of his QoS. -WARNING: - In the current implementation, memory reclaim will NOT be - triggered for a cgroup when it hits K while staying below U, which makes - this setup impractical. + .. warning:: + In the current implementation, memory reclaim will NOT be triggered for + a cgroup when it hits K while staying below U, which makes this setup + impractical. U != 0, K >= U: Since kmem charges will also be fed to the user counter and reclaim will be @@ -387,47 +394,41 @@ U != 0, K >= U: 3. User Interface ================= -3.0. Configuration ------------------- - -a. Enable CONFIG_CGROUPS -b. Enable CONFIG_MEMCG -c. Enable CONFIG_MEMCG_SWAP (to use swap extension) -d. Enable CONFIG_MEMCG_KMEM (to use kmem extension) - -3.1. Prepare the cgroups (see cgroups.txt, Why are cgroups needed?) -------------------------------------------------------------------- +To use the user interface: -:: +1. Enable CONFIG_CGROUPS and CONFIG_MEMCG options +2. Prepare the cgroups (see :ref:`Why are cgroups needed? + <cgroups-why-needed>` for the background information):: # mount -t tmpfs none /sys/fs/cgroup # mkdir /sys/fs/cgroup/memory # mount -t cgroup none /sys/fs/cgroup/memory -o memory -3.2. Make the new group and move bash into it:: +3. Make the new group and move bash into it:: # mkdir /sys/fs/cgroup/memory/0 # echo $$ > /sys/fs/cgroup/memory/0/tasks -Since now we're in the 0 cgroup, we can alter the memory limit:: +4. Since now we're in the 0 cgroup, we can alter the memory limit:: # echo 4M > /sys/fs/cgroup/memory/0/memory.limit_in_bytes -NOTE: - We can use a suffix (k, K, m, M, g or G) to indicate values in kilo, - mega or gigabytes. (Here, Kilo, Mega, Giga are Kibibytes, Mebibytes, - Gibibytes.) + The limit can now be queried:: -NOTE: - We can write "-1" to reset the ``*.limit_in_bytes(unlimited)``. + # cat /sys/fs/cgroup/memory/0/memory.limit_in_bytes + 4194304 -NOTE: - We cannot set limits on the root cgroup any more. +.. note:: + We can use a suffix (k, K, m, M, g or G) to indicate values in kilo, + mega or gigabytes. (Here, Kilo, Mega, Giga are Kibibytes, Mebibytes, + Gibibytes.) -:: +.. note:: + We can write "-1" to reset the ``*.limit_in_bytes(unlimited)``. + +.. note:: + We cannot set limits on the root cgroup any more. - # cat /sys/fs/cgroup/memory/0/memory.limit_in_bytes - 4194304 We can check the usage:: @@ -466,6 +467,8 @@ test because it has noise of shared objects/status. But the above two are testing extreme situations. Trying usual test under memory controller is always helpful. +.. _cgroup-v1-memory-test-troubleshoot: + 4.1 Troubleshooting ------------------- @@ -478,8 +481,11 @@ terminated by the OOM killer. There are several causes for this: A sync followed by echo 1 > /proc/sys/vm/drop_caches will help get rid of some of the pages cached in the cgroup (page cache pages). -To know what happens, disabling OOM_Kill as per "10. OOM Control" (below) and -seeing what happens will be helpful. +To know what happens, disabling OOM_Kill as per :ref:`"10. OOM Control" +<cgroup-v1-memory-oom-control>` (below) and seeing what happens will be +helpful. + +.. _cgroup-v1-memory-test-task-migration: 4.2 Task migration ------------------ @@ -490,26 +496,24 @@ remain charged to it, the charge is dropped when the page is freed or reclaimed. You can move charges of a task along with task migration. -See 8. "Move charges at task migration" +See :ref:`8. "Move charges at task migration" <cgroup-v1-memory-move-charges>` 4.3 Removing a cgroup --------------------- -A cgroup can be removed by rmdir, but as discussed in sections 4.1 and 4.2, a -cgroup might have some charge associated with it, even though all -tasks have migrated away from it. (because we charge against pages, not -against tasks.) +A cgroup can be removed by rmdir, but as discussed in :ref:`sections 4.1 +<cgroup-v1-memory-test-troubleshoot>` and :ref:`4.2 +<cgroup-v1-memory-test-task-migration>`, a cgroup might have some charge +associated with it, even though all tasks have migrated away from it. (because +we charge against pages, not against tasks.) -We move the stats to root (if use_hierarchy==0) or parent (if -use_hierarchy==1), and no change on the charge except uncharging +We move the stats to parent, and no change on the charge except uncharging from the child. Charges recorded in swap information is not updated at removal of cgroup. Recorded information is discarded and a cgroup which uses swap (swapcache) will be charged as a new owner of it. -About use_hierarchy, see Section 6. - 5. Misc. interfaces =================== @@ -527,76 +531,70 @@ About use_hierarchy, see Section 6. charged file caches. Some out-of-use page caches may keep charged until memory pressure happens. If you want to avoid that, force_empty will be useful. - Also, note that when memory.kmem.limit_in_bytes is set the charges due to - kernel pages will still be seen. This is not considered a failure and the - write will still return success. In this case, it is expected that - memory.kmem.usage_in_bytes == memory.usage_in_bytes. - - About use_hierarchy, see Section 6. - 5.2 stat file ------------- -memory.stat file includes following statistics - -per-memory cgroup local status -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -=============== =============================================================== -cache # of bytes of page cache memory. -rss # of bytes of anonymous and swap cache memory (includes - transparent hugepages). -rss_huge # of bytes of anonymous transparent hugepages. -mapped_file # of bytes of mapped file (includes tmpfs/shmem) -pgpgin # of charging events to the memory cgroup. The charging - event happens each time a page is accounted as either mapped - anon page(RSS) or cache page(Page Cache) to the cgroup. -pgpgout # of uncharging events to the memory cgroup. The uncharging - event happens each time a page is unaccounted from the cgroup. -swap # of bytes of swap usage -dirty # of bytes that are waiting to get written back to the disk. -writeback # of bytes of file/anon cache that are queued for syncing to - disk. -inactive_anon # of bytes of anonymous and swap cache memory on inactive - LRU list. -active_anon # of bytes of anonymous and swap cache memory on active - LRU list. -inactive_file # of bytes of file-backed memory on inactive LRU list. -active_file # of bytes of file-backed memory on active LRU list. -unevictable # of bytes of memory that cannot be reclaimed (mlocked etc). -=============== =============================================================== - -status considering hierarchy (see memory.use_hierarchy settings) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -========================= =================================================== -hierarchical_memory_limit # of bytes of memory limit with regard to hierarchy - under which the memory cgroup is -hierarchical_memsw_limit # of bytes of memory+swap limit with regard to - hierarchy under which memory cgroup is. - -total_<counter> # hierarchical version of <counter>, which in - addition to the cgroup's own value includes the - sum of all hierarchical children's values of - <counter>, i.e. total_cache -========================= =================================================== - -The following additional stats are dependent on CONFIG_DEBUG_VM -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -========================= ======================================== -recent_rotated_anon VM internal parameter. (see mm/vmscan.c) -recent_rotated_file VM internal parameter. (see mm/vmscan.c) -recent_scanned_anon VM internal parameter. (see mm/vmscan.c) -recent_scanned_file VM internal parameter. (see mm/vmscan.c) -========================= ======================================== - -Memo: +memory.stat file includes following statistics: + + * per-memory cgroup local status + + =============== =============================================================== + cache # of bytes of page cache memory. + rss # of bytes of anonymous and swap cache memory (includes + transparent hugepages). + rss_huge # of bytes of anonymous transparent hugepages. + mapped_file # of bytes of mapped file (includes tmpfs/shmem) + pgpgin # of charging events to the memory cgroup. The charging + event happens each time a page is accounted as either mapped + anon page(RSS) or cache page(Page Cache) to the cgroup. + pgpgout # of uncharging events to the memory cgroup. The uncharging + event happens each time a page is unaccounted from the + cgroup. + swap # of bytes of swap usage + swapcached # of bytes of swap cached in memory + dirty # of bytes that are waiting to get written back to the disk. + writeback # of bytes of file/anon cache that are queued for syncing to + disk. + inactive_anon # of bytes of anonymous and swap cache memory on inactive + LRU list. + active_anon # of bytes of anonymous and swap cache memory on active + LRU list. + inactive_file # of bytes of file-backed memory and MADV_FREE anonymous + memory (LazyFree pages) on inactive LRU list. + active_file # of bytes of file-backed memory on active LRU list. + unevictable # of bytes of memory that cannot be reclaimed (mlocked etc). + =============== =============================================================== + + * status considering hierarchy (see memory.use_hierarchy settings): + + ========================= =================================================== + hierarchical_memory_limit # of bytes of memory limit with regard to + hierarchy + under which the memory cgroup is + hierarchical_memsw_limit # of bytes of memory+swap limit with regard to + hierarchy under which memory cgroup is. + + total_<counter> # hierarchical version of <counter>, which in + addition to the cgroup's own value includes the + sum of all hierarchical children's values of + <counter>, i.e. total_cache + ========================= =================================================== + + * additional vm parameters (depends on CONFIG_DEBUG_VM): + + ========================= ======================================== + recent_rotated_anon VM internal parameter. (see mm/vmscan.c) + recent_rotated_file VM internal parameter. (see mm/vmscan.c) + recent_scanned_anon VM internal parameter. (see mm/vmscan.c) + recent_scanned_file VM internal parameter. (see mm/vmscan.c) + ========================= ======================================== + +.. hint:: recent_rotated means recent frequency of LRU rotation. recent_scanned means recent # of scans to LRU. showing for better debug please see the code for meanings. -Note: +.. note:: Only anonymous and swap cache memory is listed as part of 'rss' stat. This should not be confused with the true 'resident set size' or the amount of physical memory used by the cgroup. @@ -680,31 +678,20 @@ hierarchy:: d e In the diagram above, with hierarchical accounting enabled, all memory -usage of e, is accounted to its ancestors up until the root (i.e, c and root), -that has memory.use_hierarchy enabled. If one of the ancestors goes over its -limit, the reclaim algorithm reclaims from the tasks in the ancestor and the -children of the ancestor. - -6.1 Enabling hierarchical accounting and reclaim ------------------------------------------------- +usage of e, is accounted to its ancestors up until the root (i.e, c and root). +If one of the ancestors goes over its limit, the reclaim algorithm reclaims +from the tasks in the ancestor and the children of the ancestor. -A memory cgroup by default disables the hierarchy feature. Support -can be enabled by writing 1 to memory.use_hierarchy file of the root cgroup:: +6.1 Hierarchical accounting and reclaim +--------------------------------------- - # echo 1 > memory.use_hierarchy - -The feature can be disabled by:: +Hierarchical accounting is enabled by default. Disabling the hierarchical +accounting is deprecated. An attempt to do it will result in a failure +and a warning printed to dmesg. - # echo 0 > memory.use_hierarchy +For compatibility reasons writing 1 to memory.use_hierarchy will always pass:: -NOTE1: - Enabling/disabling will fail if either the cgroup already has other - cgroups created below it, or if the parent cgroup has use_hierarchy - enabled. - -NOTE2: - When panic_on_oom is set to "2", the whole system will panic in - case of an OOM event in any cgroup. + # echo 1 > memory.use_hierarchy 7. Soft limits ============== @@ -738,15 +725,25 @@ If we want to change this to 1G, we can at any time use:: # echo 1G > memory.soft_limit_in_bytes -NOTE1: +.. note:: Soft limits take effect over a long period of time, since they involve reclaiming memory for balancing between memory cgroups -NOTE2: + +.. note:: It is recommended to set the soft limit always below the hard limit, otherwise the hard limit will take precedence. -8. Move charges at task migration -================================= +.. _cgroup-v1-memory-move-charges: + +8. Move charges at task migration (DEPRECATED!) +=============================================== + +THIS IS DEPRECATED! + +It's expensive and unreliable! It's better practice to launch workload +tasks directly from inside their target cgroup. Use dedicated workload +cgroups to allow fine-grained policy adjustments without having to +move physical pages between control domains. Users can move charges associated with a task along with task migration, that is, uncharge task's pages from the old cgroup and charge them to the new cgroup. @@ -763,23 +760,29 @@ If you want to enable it:: # echo (some positive value) > memory.move_charge_at_immigrate -Note: +.. note:: Each bits of move_charge_at_immigrate has its own meaning about what type - of charges should be moved. See 8.2 for details. -Note: + of charges should be moved. See :ref:`section 8.2 + <cgroup-v1-memory-movable-charges>` for details. + +.. note:: Charges are moved only when you move mm->owner, in other words, a leader of a thread group. -Note: + +.. note:: If we cannot find enough space for the task in the destination cgroup, we try to make space by reclaiming memory. Task migration may fail if we cannot make enough space. -Note: + +.. note:: It can take several seconds if you move charges much. And if you want disable it again:: # echo 0 > memory.move_charge_at_immigrate +.. _cgroup-v1-memory-movable-charges: + 8.2 Type of charges which can be moved -------------------------------------- @@ -829,6 +832,8 @@ threshold in any direction. It's applicable for root and non-root cgroup. +.. _cgroup-v1-memory-oom-control: + 10. OOM Control =============== @@ -873,6 +878,9 @@ At reading, current status of OOM is shown. (if 1, oom-killer is disabled) - under_oom 0 or 1 (if 1, the memory cgroup is under OOM, tasks may be stopped.) + - oom_kill integer counter + The number of processes belonging to this cgroup killed by any + kind of OOM killer. 11. Memory Pressure =================== @@ -907,7 +915,7 @@ experiences some pressure. In this situation, only group C will receive the notification, i.e. groups A and B will not receive it. This is done to avoid excessive "broadcasting" of messages, which disturbs the system and which is especially bad if we are low on memory or thrashing. Group B, will receive -notification only if there are no event listers for group C. +notification only if there are no event listeners for group C. There are three optional modes that specify different propagation behavior: @@ -981,25 +989,27 @@ commented and discussed quite extensively in the community. References ========== -1. Singh, Balbir. RFC: Memory Controller, http://lwn.net/Articles/206697/ -2. Singh, Balbir. Memory Controller (RSS Control), +.. [1] Singh, Balbir. RFC: Memory Controller, http://lwn.net/Articles/206697/ +.. [2] Singh, Balbir. Memory Controller (RSS Control), http://lwn.net/Articles/222762/ -3. Emelianov, Pavel. Resource controllers based on process cgroups - http://lkml.org/lkml/2007/3/6/198 -4. Emelianov, Pavel. RSS controller based on process cgroups (v2) - http://lkml.org/lkml/2007/4/9/78 -5. Emelianov, Pavel. RSS controller based on process cgroups (v3) - http://lkml.org/lkml/2007/5/30/244 +.. [3] Emelianov, Pavel. Resource controllers based on process cgroups + https://lore.kernel.org/r/45ED7DEC.7010403@sw.ru +.. [4] Emelianov, Pavel. RSS controller based on process cgroups (v2) + https://lore.kernel.org/r/461A3010.90403@sw.ru +.. [5] Emelianov, Pavel. RSS controller based on process cgroups (v3) + https://lore.kernel.org/r/465D9739.8070209@openvz.org + 6. Menage, Paul. Control Groups v10, http://lwn.net/Articles/236032/ 7. Vaidyanathan, Srinivasan, Control Groups: Pagecache accounting and control subsystem (v3), http://lwn.net/Articles/235534/ 8. Singh, Balbir. RSS controller v2 test results (lmbench), - http://lkml.org/lkml/2007/5/17/232 + https://lore.kernel.org/r/464C95D4.7070806@linux.vnet.ibm.com 9. Singh, Balbir. RSS controller v2 AIM9 results - http://lkml.org/lkml/2007/5/18/1 + https://lore.kernel.org/r/464D267A.50107@linux.vnet.ibm.com 10. Singh, Balbir. Memory controller v6 test results, - http://lkml.org/lkml/2007/8/19/36 -11. Singh, Balbir. Memory controller introduction (v6), - http://lkml.org/lkml/2007/8/17/69 -12. Corbet, Jonathan, Controlling memory use in cgroups, - http://lwn.net/Articles/243795/ + https://lore.kernel.org/r/20070819094658.654.84837.sendpatchset@balbir-laptop + +.. [11] Singh, Balbir. Memory controller introduction (v6), + https://lore.kernel.org/r/20070817084228.26003.12568.sendpatchset@balbir-laptop +.. [12] Corbet, Jonathan, Controlling memory use in cgroups, + http://lwn.net/Articles/243795/ diff --git a/Documentation/admin-guide/cgroup-v1/misc.rst b/Documentation/admin-guide/cgroup-v1/misc.rst new file mode 100644 index 000000000000..661614c24df3 --- /dev/null +++ b/Documentation/admin-guide/cgroup-v1/misc.rst @@ -0,0 +1,4 @@ +=============== +Misc controller +=============== +Please refer "Misc" documentation in Documentation/admin-guide/cgroup-v2.rst diff --git a/Documentation/admin-guide/cgroup-v1/rdma.rst b/Documentation/admin-guide/cgroup-v1/rdma.rst index 2fcb0a9bf790..e69369b7252e 100644 --- a/Documentation/admin-guide/cgroup-v1/rdma.rst +++ b/Documentation/admin-guide/cgroup-v1/rdma.rst @@ -114,4 +114,4 @@ Following resources can be accounted by rdma controller. (d) Delete resource limit:: - echo echo mlx4_0 hca_handle=max hca_object=max > /sys/fs/cgroup/rdma/1/rdma.max + echo mlx4_0 hca_handle=max hca_object=max > /sys/fs/cgroup/rdma/1/rdma.max |