Problem with lxcfs 4.0.1 / online CPU is not working

[Linyu]

Hi forum,
I've noticed that lxcfs has virtualized the /sys/devices/system/cpu/online dir since lxcfs version 3.1.2, but it's not working for proxmox LXC:

After upgrade the proxmox, I saw the lxcfs version has upgrade to 4.0.1, but it seems not working:

Host:

root@node-rcc-gd1:/etc/apt/sources.list.d# pct cpusets
----------------------------------------
103:                                  14
----------------------------------------

CT:

[root@CT103 ~]# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                32
On-line CPU(s) list:   0-31
...

I need this function because in python, psutil.cpu_count() will return the cores count of host, which was not what I want.
BTW, in some app, like Nginx, the configuration file "worker_process auto;" will cause huge lag because the nginx can not recognize the cpu cores, it will needs to change it by our own.

Virtualize /sys/devices/system/cpu/online
LXCFS now also partially virtualizes sysfs. The first file to virtualize is /sys/devices/system/cpu/online per container.

See:https://discuss.linuxcontainers.org/t/lxcfs-3-1-2-has-been-released/5321

pveversion -v:

root@node-rcc-gd1:/etc/apt/sources.list.d# pveversion -v
proxmox-ve: 6.1-2 (running kernel: 5.3.13-2-pve)
pve-manager: 6.1-8 (running version: 6.1-8/806edfe1)
pve-kernel-helper: 6.1-7
pve-kernel-5.3: 6.1-6
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.13-2-pve: 5.3.13-2
ceph-fuse: 14.2.8-pve1
corosync: 3.0.3-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 2.0.1-1+pve8
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.15-pve1
libpve-access-control: 6.0-6
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.0-17
libpve-guest-common-perl: 3.0-5
libpve-http-server-perl: 3.0-5
libpve-storage-perl: 6.1-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 3.2.1-1
lxcfs: 4.0.1-pve1

 

[Linyu]

Another CT:

[root@RainYunRDMAWisq ~]# cat /sys/devices/system/cpu/online
0-31

Any idea?

 

[wbumiller]

That functionality is not enabled by default. You need to modify the lxcfs.service to pass `--enable-cfs`

# /etc/systemd/system/lxcfs.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/lxcfs --enable-cfs /var/lib/lxcfs

<rant mode>
Code which tries to spawn processes based on the available cores/threads should better use `sched_getaffinity(2)` to get the cores the system actually allows it to use, rather than reading arbitrary system files relating to physical information instead of process scheduling. (But yeah, I know that's not what most people do for some reason...)
</rant mode>

 

[Linyu]

That functionality is not enabled by default. You need to modify the lxcfs.service to pass `--enable-cfs`

# /etc/systemd/system/lxcfs.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/lxcfs --enable-cfs /var/lib/lxcfs

<rant mode>
Code which tries to spawn processes based on the available cores/threads should better use `sched_getaffinity(2)` to get the cores the system actually allows it to use, rather than reading arbitrary system files relating to physical information instead of process scheduling. (But yeah, I know that's not what most people do for some reason...)
</rant mode>

Thank you very much, this will help me a lot.

 

[Linyu]

That functionality is not enabled by default. You need to modify the lxcfs.service to pass `--enable-cfs`

# /etc/systemd/system/lxcfs.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/lxcfs --enable-cfs /var/lib/lxcfs

<rant mode>
Code which tries to spawn processes based on the available cores/threads should better use `sched_getaffinity(2)` to get the cores the system actually allows it to use, rather than reading arbitrary system files relating to physical information instead of process scheduling. (But yeah, I know that's not what most people do for some reason...)
</rant mode>

I tested the code, but it cause another problem:
The CT and recognize all cpu usage of the host which is not possible before change the setting.

 

[Linyu]

Online CPU works great:

 

[wbumiller]

Right, looks like that's the case. The thing is, these options change a few things around, and depending on what strategy the tools such as htop use to get their information, they may see more or less. Isolating these things properly is next to impossible. We can look into this more soon. (Note that this is not pve-specific.)