From: Vasily Averin <vvs@openvz.org>
To: "Michal Koutný" <mkoutny@suse.com>
Cc: Roman Gushchin <roman.gushchin@linux.dev>,
Shakeel Butt <shakeelb@google.com>,
kernel@openvz.org, linux-kernel@vger.kernel.org,
Vlastimil Babka <vbabka@suse.cz>, Michal Hocko <mhocko@suse.com>,
cgroups@vger.kernel.org
Subject: Re: [PATCH 3/4] memcg: enable accounting for struct cgroup
Date: Fri, 20 May 2022 23:16:32 +0300 [thread overview]
Message-ID: <d28233ee-bccb-7bc3-c2ec-461fd7f95e6a@openvz.org> (raw)
In-Reply-To: <740dfcb1-5c5f-6a40-0f71-65f277f976d6@openvz.org>
On 5/20/22 10:24, Vasily Averin wrote:
> On 5/19/22 19:53, Michal Koutný wrote:
>> On Fri, May 13, 2022 at 06:52:12PM +0300, Vasily Averin <vvs@openvz.org> wrote:
>>> Creating each new cgroup allocates 4Kb for struct cgroup. This is the
>>> largest memory allocation in this scenario and is epecially important
>>> for small VMs with 1-2 CPUs.
>>
>> What do you mean by this argument?
>>
>> (On bigger irons, the percpu components becomes dominant, e.g. struct
>> cgroup_rstat_cpu.)
>
> Michal, Shakeel,
> thank you very much for your feedback, it helps me understand how to improve
> the methodology of my accounting analyze.
> I considered the general case and looked for places of maximum memory allocations.
> Now I think it would be better to split all called allocations into:
> - common part, called for any cgroup type (i.e. cgroup_mkdir and cgroup_create),
> - per-cgroup parts,
> and focus on 2 corner cases: for single CPU VMs and for "big irons".
> It helps to clarify which allocations are accounting-important and which ones
> can be safely ignored.
>
> So right now I'm going to redo the calculations and hope it doesn't take long.
common part: ~11Kb + 318 bytes percpu
memcg: ~17Kb + 4692 bytes percpu
cpu: ~2.5Kb + 1036 bytes percpu
cpuset: ~3Kb + 12 bytes percpu
blkcg: ~3Kb + 12 bytes percpu
pid: ~1.5Kb + 12 bytes percpu
perf: ~320b + 60 bytes percpu
-------------------------------------------
total: ~38Kb + 6142 bytes percpu
currently accounted: 4668 bytes percpu
Results:
a) I'll add accounting for cgroup_rstat_cpu and psi_group_cpu,
they are called in common part and consumes 288 bytes percpu.
b) It makes sense to add accounting for simple_xattr(), as Michal recommend,
especially because it can grow over 4kb
c) it looks like the rest of the allocations can be ignored
Details are below
('=' -- already accounted, '+' -- to be accounted, '~' -- see KERNFS, '?' -- perhaps later )
common part:
16 ~ 352 5632 5632 KERNFS (*)
1 + 4096 4096 9728 (cgroup_mkdir+0xe4)
1 584 584 10312 (radix_tree_node_alloc.constprop.0+0x89)
1 192 192 10504 (__d_alloc+0x29)
2 72 144 10648 (avc_alloc_node+0x27)
2 64 128 10776 (percpu_ref_init+0x6a)
1 64 64 10840 (memcg_list_lru_alloc+0x21a)
1 + 192 192 192 call_site=psi_cgroup_alloc+0x1e
1 + 96 96 288 call_site=cgroup_rstat_init+0x5f
2 12 24 312 call_site=percpu_ref_init+0x23
1 6 6 318 call_site=__percpu_counter_init+0x22
(*) KERNFS includes:
1 + 128 (__kernfs_new_node+0x4d) kernfs node
1 + 88 (__kernfs_iattrs+0x57) kernfs iattrs
1 + 96 (simple_xattr_alloc+0x28) simple_xattr_alloc() that can grow over 4Kb
1 ? 32 (simple_xattr_set+0x59)
1 8 (__kernfs_new_node+0x30)
memory:
------
1 + 8192 8192 8192 (mem_cgroup_css_alloc+0x4a)
14 ~ 352 4928 13120 KERNFS
1 + 2048 2048 15168 (mem_cgroup_css_alloc+0xdd)
1 1024 1024 16192 (alloc_shrinker_info+0x79)
1 584 584 16776 (radix_tree_node_alloc.constprop.0+0x89)
2 64 128 16904 (percpu_ref_init+0x6a)
1 64 64 16968 (mem_cgroup_css_online+0x32)
1 = 3684 3684 3684 call_site=mem_cgroup_css_alloc+0x9e
1 = 984 984 4668 call_site=mem_cgroup_css_alloc+0xfd
2 12 24 4692 call_site=percpu_ref_init+0x23
cpu:
---
5 ~ 352 1760 1760 KERNFS
1 640 640 2400 (sched_create_group+0x1b)
1 64 64 2464 (percpu_ref_init+0x6a)
1 32 32 2496 (alloc_fair_sched_group+0x55)
1 32 32 2528 (alloc_fair_sched_group+0x31)
4 + 512 512 512 (alloc_fair_sched_group+0x16c)
4 + 512 512 1024 (alloc_fair_sched_group+0x13e)
1 12 12 1036 call_site=percpu_ref_init+0x23
cpuset:
------
5 ~ 352 1760 1760 KERNFS
1 1024 1024 2784 (cpuset_css_alloc+0x2f)
1 64 64 2848 (percpu_ref_init+0x6a)
3 8 24 2872 (alloc_cpumask_var_node+0x1f)
1 12 12 12 call_site=percpu_ref_init+0x23
blkcg:
-----
6 ~ 352 2112 2112 KERNFS
1 512 512 2624 (blkcg_css_alloc+0x37)
1 64 64 2688 (percpu_ref_init+0x6a)
1 32 32 2720 (ioprio_alloc_cpd+0x39)
1 32 32 2752 (ioc_cpd_alloc+0x39)
1 32 32 2784 (blkcg_css_alloc+0x66)
1 12 12 12 call_site=percpu_ref_init+0x23
pid:
---
3 ~ 352 1056 1056 KERNFS
1 512 512 1568 (pids_css_alloc+0x1b)
1 64 64 1632 (percpu_ref_init+0x6a)
1 12 12 12 call_site=percpu_ref_init+0x23
perf:
----
1 256 256 256 (perf_cgroup_css_alloc+0x1c)
1 64 64 320 (percpu_ref_init+0x6a)
1 48 48 48 call_site=perf_cgroup_css_alloc+0x33
1 12 12 60 call_site=percpu_ref_init+0x23
Thank you,
Vasily Averin
next prev parent reply other threads:[~2022-05-20 20:16 UTC|newest]
Thread overview: 139+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-27 10:37 [PATCH] memcg: accounting for objects allocated for new netdevice Vasily Averin
2022-04-27 14:01 ` Michal Koutný
2022-04-27 16:52 ` Shakeel Butt
2022-04-27 22:35 ` Vasily Averin
2022-05-02 12:15 ` [PATCH memcg v2] " Vasily Averin
2022-05-04 20:50 ` Luis Chamberlain
2022-05-05 3:50 ` patchwork-bot+netdevbpf
2022-05-11 2:51 ` Roman Gushchin
2022-05-02 19:37 ` kernfs memcg accounting Vasily Averin
2022-05-02 21:22 ` Michal Koutný
2022-05-04 9:00 ` Vasily Averin
2022-05-04 14:10 ` Michal Koutný
2022-05-04 21:16 ` Vasily Averin
2022-05-05 9:47 ` Michal Koutný
2022-05-06 8:37 ` Vasily Averin
2022-05-11 3:06 ` Roman Gushchin
2022-05-11 6:01 ` Vasily Averin
2022-05-11 16:49 ` Michal Koutný
2022-05-11 17:46 ` Roman Gushchin
2022-05-11 16:34 ` Michal Koutný
2022-05-11 18:10 ` Roman Gushchin
2022-05-13 15:51 ` [PATCH 0/4] memcg: accounting for objects allocated by mkdir cgroup Vasily Averin
2022-05-13 17:49 ` Roman Gushchin
2022-05-21 16:37 ` [PATCH mm v2 0/9] " Vasily Averin
2022-05-30 11:25 ` [PATCH mm v3 " Vasily Averin
2022-05-30 11:55 ` Michal Hocko
2022-05-30 13:09 ` Vasily Averin
2022-05-30 14:22 ` Michal Hocko
2022-05-30 19:58 ` Vasily Averin
2022-05-31 7:16 ` Michal Hocko
2022-06-01 3:43 ` Vasily Averin
2022-06-01 9:15 ` Michal Koutný
2022-06-01 9:32 ` Michal Hocko
2022-06-01 13:05 ` Michal Hocko
2022-06-01 14:22 ` Roman Gushchin
2022-06-01 15:24 ` Michal Hocko
2022-06-01 9:26 ` Michal Hocko
2022-06-13 5:34 ` [PATCH mm v4 " Vasily Averin
2022-06-23 14:50 ` [PATCH mm v5 0/9] memcg: accounting for objects allocated by mkdir, cgroup Vasily Averin
2022-06-23 15:03 ` Vasily Averin
2022-06-23 16:07 ` Michal Hocko
2022-06-23 16:55 ` Shakeel Butt
2022-06-24 10:40 ` Vasily Averin
2022-06-24 12:26 ` Michal Koutný
2022-06-24 13:59 ` Michal Hocko
2022-06-25 9:43 ` [PATCH RFC] memcg: avoid idr ids space depletion Vasily Averin
2022-06-25 14:04 ` [PATCH RFC] memcg: notify about global mem_cgroup_id " Vasily Averin
2022-06-26 1:56 ` Roman Gushchin
2022-06-26 7:11 ` Vasily Averin
2022-06-27 2:12 ` [PATCH cgroup] cgroup: set the correct return code if hierarchy limits are reached Vasily Averin
2022-06-27 3:33 ` Muchun Song
2022-06-27 9:07 ` Tejun Heo
2022-06-28 0:44 ` Roman Gushchin
2022-06-28 3:59 ` Vasily Averin
2022-06-28 9:16 ` Michal Koutný
2022-06-28 9:22 ` Tejun Heo
2022-06-29 6:13 ` Vasily Averin
2022-06-29 19:25 ` Tejun Heo
2022-07-01 2:42 ` Roman Gushchin
2022-06-27 2:11 ` [PATCH mm v2] memcg: notify about global mem_cgroup_id space depletion Vasily Averin
2022-06-27 3:23 ` Muchun Song
2022-06-27 6:49 ` Vasily Averin
2022-06-28 1:11 ` Roman Gushchin
2022-06-28 3:43 ` Vasily Averin
2022-06-28 9:08 ` Michal Koutný
2022-06-27 16:37 ` [PATCH mm v5 0/9] memcg: accounting for objects allocated by mkdir, cgroup Shakeel Butt
2022-07-01 11:03 ` Michal Hocko
2022-07-10 18:53 ` Vasily Averin
2022-07-11 16:24 ` Michal Hocko
2022-06-23 14:50 ` [PATCH mm v5 1/9] memcg: enable accounting for struct cgroup Vasily Averin
2022-06-23 14:50 ` [PATCH mm v5 2/9] memcg: enable accounting for kernfs nodes Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 3/9] memcg: enable accounting for kernfs iattrs Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 4/9] memcg: enable accounting for struct simple_xattr Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 5/9] memcg: enable accounting for percpu allocation of struct psi_group_cpu Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 6/9] memcg: enable accounting for percpu allocation of struct cgroup_rstat_cpu Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 7/9] memcg: enable accounting for large allocations in mem_cgroup_css_alloc Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 8/9] memcg: enable accounting for allocations in alloc_fair_sched_group Vasily Averin
2022-06-23 14:52 ` [PATCH mm v5 9/9] memcg: enable accounting for perpu allocation of struct rt_rq Vasily Averin
2022-06-13 5:34 ` [PATCH mm v4 1/9] memcg: enable accounting for struct cgroup Vasily Averin
2022-06-13 5:34 ` [PATCH mm v4 2/9] memcg: enable accounting for kernfs nodes Vasily Averin
2022-06-13 5:34 ` [PATCH mm v4 3/9] memcg: enable accounting for kernfs iattrs Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 4/9] memcg: enable accounting for struct simple_xattr Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 5/9] memcg: enable accounting for percpu allocation of struct psi_group_cpu Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 6/9] memcg: enable accounting for percpu allocation of struct cgroup_rstat_cpu Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 7/9] memcg: enable accounting for large allocations in mem_cgroup_css_alloc Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 8/9] memcg: enable accounting for allocations in alloc_fair_sched_group Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 9/9] memcg: enable accounting for perpu allocation of struct rt_rq Vasily Averin
[not found] ` <cover.1653899364.git.vvs@openvz.org>
2022-05-30 11:25 ` [PATCH mm v3 1/9] memcg: enable accounting for struct cgroup Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 2/9] memcg: enable accounting for kernfs nodes Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 3/9] memcg: enable accounting for kernfs iattrs Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 4/9] memcg: enable accounting for struct simple_xattr Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 5/9] memcg: enable accounting for percpu allocation of struct psi_group_cpu Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 6/9] memcg: enable accounting for percpu allocation of struct cgroup_rstat_cpu Vasily Averin
2022-05-30 15:04 ` Muchun Song
2022-05-30 11:26 ` [PATCH mm v3 7/9] memcg: enable accounting for large allocations in mem_cgroup_css_alloc Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 8/9] memcg: enable accounting for allocations in alloc_fair_sched_group Vasily Averin
2022-05-30 11:27 ` [PATCH mm v3 9/9] memcg: enable accounting for perpu allocation of struct rt_rq Vasily Averin
2022-05-30 15:06 ` Muchun Song
2022-05-21 16:37 ` [PATCH mm v2 1/9] memcg: enable accounting for struct cgroup Vasily Averin
2022-05-22 6:37 ` Muchun Song
2022-05-21 16:37 ` [PATCH mm v2 2/9] memcg: enable accounting for kernfs nodes Vasily Averin
2022-05-22 6:37 ` Muchun Song
2022-05-21 16:37 ` [PATCH mm v2 3/9] memcg: enable accounting for kernfs iattrs Vasily Averin
2022-05-22 6:38 ` Muchun Song
2022-05-21 16:38 ` [PATCH mm v2 4/9] memcg: enable accounting for struct simple_xattr Vasily Averin
2022-05-22 6:38 ` Muchun Song
2022-05-21 16:38 ` [PATCH mm v2 5/9] memcg: enable accounting for percpu allocation of struct psi_group_cpu Vasily Averin
2022-05-21 21:34 ` Shakeel Butt
2022-05-22 6:40 ` Muchun Song
2022-05-25 1:30 ` Roman Gushchin
2022-05-21 16:38 ` [PATCH mm v2 6/9] memcg: enable accounting for percpu allocation of struct cgroup_rstat_cpu Vasily Averin
2022-05-21 17:58 ` Vasily Averin
2022-05-21 21:35 ` Shakeel Butt
2022-05-21 22:05 ` kernel test robot
2022-05-25 1:31 ` Roman Gushchin
2022-05-21 16:38 ` [PATCH mm v2 7/9] memcg: enable accounting for large allocations in mem_cgroup_css_alloc Vasily Averin
2022-05-22 6:47 ` Muchun Song
2022-05-21 16:38 ` [PATCH mm v2 8/9] memcg: enable accounting for allocations in alloc_fair_sched_group Vasily Averin
2022-05-22 6:49 ` Muchun Song
2022-05-21 16:39 ` [PATCH mm v2 9/9] memcg: enable accounting for percpu allocation of struct rt_rq Vasily Averin
2022-05-21 21:37 ` Shakeel Butt
2022-05-25 1:31 ` Roman Gushchin
2022-05-13 15:51 ` [PATCH 1/4] memcg: enable accounting for large allocations in mem_cgroup_css_alloc Vasily Averin
2022-05-19 16:46 ` Michal Koutný
2022-05-20 1:07 ` Shakeel Butt
2022-05-13 15:51 ` [PATCH 2/4] memcg: enable accounting for kernfs nodes and iattrs Vasily Averin
2022-05-19 16:33 ` Michal Koutný
2022-05-20 1:12 ` Shakeel Butt
2022-05-13 15:52 ` [PATCH 3/4] memcg: enable accounting for struct cgroup Vasily Averin
2022-05-19 16:53 ` Michal Koutný
2022-05-20 7:24 ` Vasily Averin
2022-05-20 20:16 ` Vasily Averin [this message]
2022-05-21 0:55 ` Roman Gushchin
2022-05-21 7:28 ` Vasily Averin
2022-05-23 13:52 ` Michal Koutný
2022-05-20 1:31 ` Shakeel Butt
2022-05-13 15:52 ` [PATCH 4/4] memcg: enable accounting for allocations in alloc_fair_sched_group Vasily Averin
2022-05-19 16:45 ` Michal Koutný
2022-05-20 1:18 ` Shakeel Butt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d28233ee-bccb-7bc3-c2ec-461fd7f95e6a@openvz.org \
--to=vvs@openvz.org \
--cc=cgroups@vger.kernel.org \
--cc=kernel@openvz.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mhocko@suse.com \
--cc=mkoutny@suse.com \
--cc=roman.gushchin@linux.dev \
--cc=shakeelb@google.com \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Powered by JetHome