LXC containers shows host's load average

[decibel83]

Hi,
my LXC containers are showing the same load average of the host.
I know this is due to some limitations of LXC and I had the same problem with CPU and memory too until some months ago.
Now after some upgrades containers are showing the correct information about CPU and memory limits. Why it's not the same for load average too?

The container 911 (staging3) is running on node9:

root@node09:~# pct list
VMID       Status     Lock         Name               
904        running                 mediaspooler1       
905        running                 mediaspooler2       
909        running                 staging4           
910        running                 staging2           
911        running                 staging3           
1105       running                 wifi

Load average on node9 is 1.37, 1.73, 1.46:

root@node09:~# w
 08:33:25 up 171 days, 10:39,  1 user,  load average: 1.37, 1.73, 1.46
[...]

Load average on staging3 is 1.37, 1.73, 1.46:

root@staging3:/# w
 08:33:24 up 1 day, 20:34,  1 user,  load average: 1.37, 1.73, 1.46
[...]

I verified that if host's load average goes up, also the container's one grows, so they are synchronized.

Could you help me please?
Thanks!

 

[sahostking]

Its related to this issue that was fixed only recently: https://github.com/lxc/lxcfs/issues/13

It should appear in proxmox sometime soon

 

[dbartus]

any idea when the fix will be applied to proxmox?

 

[fabian]

filed https://bugzilla.proxmox.com/show_bug.cgi?id=1870 for tracking on our side.

 

[dbartus]

Hi Proxmox team,

I've been checking the bugzilla for a while on this issue, and it's still on status "Assigned". Any update on this issue?

I've also seen there are few other threads on this in this forum as well.

Thanks for your reply. Really looking forward for this feature to be implemented on Proxmox.

 

[mailinglists]

By the looks of things in mailing list it seems like this patch was included today.

 

[oguz]

This is now resolved. Whether to enable it default or not, will be decided later.

Willing users can enable this behaviour by editing:

/lib/systemd/system/lxcfs.service

and by adding the `-l` flag in ExecStart. Then restart service and containers.

 

[jmcorrea]

Hello!
we have 2 hosts running Proxmox 5.2.1. I checked and lxc-pve is version 3.0.0-3. Did a apt-get upgrade and rebooted server, after that added the -l to /lib/systemd/system/lxcfs.service, but still top shows the host load.
Do we need to upgrade anything else?
Thanks!
Juan Correa

 

[oguz]

Hello!
we have 2 hosts running Proxmox 5.2.1. I checked and lxc-pve is version 3.0.0-3. Did a apt-get upgrade and rebooted server, after that added the -l to /lib/systemd/system/lxcfs.service, but still top shows the host load.

The package is `lxcfs` and the version with this patch is 3.0.3-pve1

Did you do `apt update` before `apt upgrade`? Was the lxcfs package upgraded at all? Which repository are you using? This is not in the enterprise repository yet, but you should be able to find it in pve-no-subscription

 

[jmcorrea]

Hello Oguz,

thank you! I just noticed that I didnt have the no-subscription repositories enabled. I added them, updated lxcfs, added the -l to
/lib/systemd/system/lxcfs.service and now the load is showing correctly.

Best regards,
Juan Correa

 

[oguz]

I just noticed that I didnt have the no-subscription repositories enabled.

Which repository were you using? Use no-subscription repository only if you don't use the enterprise repositories. It will cause package management problems to have both repositories.

https://pve.proxmox.com/wiki/Package_Repositories

 

[jmcorrea]

Hello Oguz,

we have the enterprise repositories enabled, but we don't have a subscription yet, we are going to buy it but im waiting for my boss to do it. I commented the enterprise repository and enabled the no-suscription ones. Run a apt clean, apt update and apt upgrade lxcfs. After that I undid the changed made to the repositories, so we are back to the enterprise ones.

Thanks!

Juan Correa

 

[oguz]

we have the enterprise repositories enabled, but we don't have a subscription yet, we are going to buy it but im waiting for my boss to do it. I commented the enterprise repository and enabled the no-suscription ones.

How are you using the enterprise repository without a subscription? It should be returning a 401 Unauthorized error if you didn't enable subscription yet (which means you are not going to be able to get any updates).

I suggest you to switch to the pve-no-subscription repository and do a

apt update
apt full-upgrade

to get the latest version from everything. You can re-enable the enterprise repositories once you get your subscription.

 

[jmcorrea]

Hello Oguz,

thanks! Indeed the updates get 401 error. I saw that the no-subscription packages has a warning on the docs:
It can be used for testing and non-production use. Its not recommended to run on production servers, as these packages are not always heavily tested and validated.
Our servers are in production and they already have quite a few servers. That's why I only updated lxcfs instead of a full upgrade.

Anyway, we should be getting the subscription in a few days, we have it on our to-do list, so I guess its safer to wait and upgrade with the enterprise safe and tested packages.

Best regards,

Juan Correa

 

[kotakomputer]

# pveversion
pve-manager/5.4-13/aee6f0ec (running kernel: 4.15.18-18-pve)
#
# apt-get upgrade lxcfs
Reading package lists... Done
Building dependency tree
Reading state information... Done
lxcfs is already the newest version (3.0.3-pve1).

I have edited "ExecStart=/usr/bin/lxcfs -l /var/lib/lxcfs/", restart lxc and lxcfs service then restart all containers.
But CT still shows load of host. Any idea?

 

[Norberto Iannicelli]

I have the same problem in my LXC machine.
the container get load average to my node on htop the lxc container.

proxmox-ve: 5.4-2 (running kernel: 4.15.18-21-pve)
pve-manager: 5.4-13 (running version: 5.4-13/aee6f0ec)
pve-kernel-4.15: 5.4-9
pve-kernel-4.15.18-21-pve: 4.15.18-47
pve-kernel-4.15.18-20-pve: 4.15.18-46
pve-kernel-4.15.18-17-pve: 4.15.18-43
pve-kernel-4.15.18-12-pve: 4.15.18-36
ceph: 12.2.12-pve1
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-12
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-54
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-14
libpve-storage-perl: 5.0-44
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-7
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-38
pve-container: 2.0-40
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-7
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-4
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-54
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3

 

[jmcorrea]

Hello!

perhaps this can be useful. Our servers are running the same pve-manager and lxcfs as yours, and after the change to the /lib/systemd/system/lxcfs.service they still give the correct load.

root@server:~# pveversion
pve-manager/5.4-13/aee6f0ec (running kernel: 4.15.18-20-pve)
root@server:~# apt-get upgrade lxcfs
Reading package lists... Done
Building dependency tree       
Reading state information... Done
lxcfs is already the newest version (3.0.3-pve1).

This is the /lib/systemd/system/lxcfs.service file:

[Unit]
Description=FUSE filesystem for LXC
ConditionVirtualization=!container
Before=lxc.service

[Service]
ExecStart=/usr/bin/lxcfs -l /var/lib/lxcfs/
KillMode=process
Restart=on-failure
ExecReload=/bin/kill -USR1 $MAINPID
ExecStopPost=-/bin/fusermount -u /var/lib/lxcfs

[Install]
WantedBy=multi-user.target

Hope it can help.

Best regards,

Juan.

 

[kotakomputer]

Hello!

perhaps this can be useful. Our servers are running the same pve-manager and lxcfs as yours, and after the change to the /lib/systemd/system/lxcfs.service they still give the correct load.



Hope it can help.

Best regards,

Juan.

Great Juan ... I just recheck that #top command reported right load, but WHM/CPanel (Home »System Health »Process Manager) still report node load instead

 

[Andrei ZeeGiant]

Is this still an issue on Proxmox 6, and the cPanel "Process Manager" showing the host load, while 'uptime' and top/htop show the lxc container load? I'm in the process of onboarding a new client and would rather put them on a cPanel lxc container vs kvm. Thanks

 

[Andrei ZeeGiant]

Is this still an issue on Proxmox 6, and the cPanel "Process Manager" showing the host load, while 'uptime' and top/htop show the lxc container load? I'm in the process of onboarding a new client and would rather put them on a cPanel lxc container vs kvm. Thanks

Confirmed still an issue on cPanel 72 through 84. While 'uptime', service httpd fullstatus and /proc/loadavg all report the LXC load, the WHM "load average" seen in the top right header still reflects the VPS host node value. I opened a ticket with them to see if they can change the method in which they're pulling the load value or possibly share their method with the community in order for someone at OpenVZ or Proxmox to possibly patch it upstream.