Problem with lxcfs 4.0.1 / online CPU is not working

Linyu

Member
Jun 14, 2019
36
1
13
23
Hi forum,
I've noticed that lxcfs has virtualized the /sys/devices/system/cpu/online dir since lxcfs version 3.1.2, but it's not working for proxmox LXC:

After upgrade the proxmox, I saw the lxcfs version has upgrade to 4.0.1, but it seems not working:

Host:
Code:
root@node-rcc-gd1:/etc/apt/sources.list.d# pct cpusets
----------------------------------------
103:                                  14
----------------------------------------

CT:
Code:
[root@CT103 ~]# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                32
On-line CPU(s) list:   0-31
...

I need this function because in python, psutil.cpu_count() will return the cores count of host, which was not what I want.
BTW, in some app, like Nginx, the configuration file "worker_process auto;" will cause huge lag because the nginx can not recognize the cpu cores, it will needs to change it by our own.


Virtualize /sys/devices/system/cpu/online
LXCFS now also partially virtualizes sysfs. The first file to virtualize is /sys/devices/system/cpu/online per container.

See:https://discuss.linuxcontainers.org/t/lxcfs-3-1-2-has-been-released/5321

pveversion -v:
Code:
root@node-rcc-gd1:/etc/apt/sources.list.d# pveversion -v
proxmox-ve: 6.1-2 (running kernel: 5.3.13-2-pve)
pve-manager: 6.1-8 (running version: 6.1-8/806edfe1)
pve-kernel-helper: 6.1-7
pve-kernel-5.3: 6.1-6
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.13-2-pve: 5.3.13-2
ceph-fuse: 14.2.8-pve1
corosync: 3.0.3-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 2.0.1-1+pve8
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.15-pve1
libpve-access-control: 6.0-6
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.0-17
libpve-guest-common-perl: 3.0-5
libpve-http-server-perl: 3.0-5
libpve-storage-perl: 6.1-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 3.2.1-1
lxcfs: 4.0.1-pve1
 

Linyu

Member
Jun 14, 2019
36
1
13
23
Another CT:
Code:
[root@RainYunRDMAWisq ~]# cat /sys/devices/system/cpu/online
0-31
Any idea?
 

wbumiller

Proxmox Staff Member
Staff member
Jun 23, 2015
714
122
63
That functionality is not enabled by default. You need to modify the lxcfs.service to pass `--enable-cfs`
Code:
# /etc/systemd/system/lxcfs.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/lxcfs --enable-cfs /var/lib/lxcfs

<rant mode>
Code which tries to spawn processes based on the available cores/threads should better use `sched_getaffinity(2)` to get the cores the system actually allows it to use, rather than reading arbitrary system files relating to physical information instead of process scheduling. (But yeah, I know that's not what most people do for some reason...)
</rant mode>
 
  • Like
Reactions: Linyu

Linyu

Member
Jun 14, 2019
36
1
13
23
That functionality is not enabled by default. You need to modify the lxcfs.service to pass `--enable-cfs`
Code:
# /etc/systemd/system/lxcfs.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/lxcfs --enable-cfs /var/lib/lxcfs

<rant mode>
Code which tries to spawn processes based on the available cores/threads should better use `sched_getaffinity(2)` to get the cores the system actually allows it to use, rather than reading arbitrary system files relating to physical information instead of process scheduling. (But yeah, I know that's not what most people do for some reason...)
</rant mode>
Thank you very much, this will help me a lot.
 

Linyu

Member
Jun 14, 2019
36
1
13
23
That functionality is not enabled by default. You need to modify the lxcfs.service to pass `--enable-cfs`
Code:
# /etc/systemd/system/lxcfs.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/lxcfs --enable-cfs /var/lib/lxcfs

<rant mode>
Code which tries to spawn processes based on the available cores/threads should better use `sched_getaffinity(2)` to get the cores the system actually allows it to use, rather than reading arbitrary system files relating to physical information instead of process scheduling. (But yeah, I know that's not what most people do for some reason...)
</rant mode>
I tested the code, but it cause another problem:
The CT and recognize all cpu usage of the host which is not possible before change the setting.
20200406180022.png
QQ截图20200406180159.png
 

wbumiller

Proxmox Staff Member
Staff member
Jun 23, 2015
714
122
63
Right, looks like that's the case. The thing is, these options change a few things around, and depending on what strategy the tools such as htop use to get their information, they may see more or less. Isolating these things properly is next to impossible. We can look into this more soon. (Note that this is not pve-specific.)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!