[SOLVED] pve7to8 cgroup warning incorrect?

[shukar]

With the announcement of the v8 beta I figured I'd get a jump on ensuring I'm ready by running the pve7to8 tool on my system.

The regular run came through clean, but when I run it with --full I get a warning:

Found at least one CT (###) which does not support running in a unified cgroup v2 layout
    Consider upgrading the Containers distro or set systemd.unified_cgroup_hierarchy=0 in the Proxmox VE hosts' kernel cmdline! Skipping further CT compat checks.

Which I don't think is accurate given that the container in question was created on v7, includes no group entries in the configuration or any pass through of devices. The container in question is running Debian 12, and is running systemd 252.6-1, which should be appropriately compatible. Is this a bug in the tool I should report, or is there something I'm potentially missing that could be causing the error?

Please let me know if there's any additional data about my setup or the container that is needed to determine potential causes. Thanks!

 

[Chris]

Hi,
thanks for bringing this to our attention. There was indeed an issue with the systemd version check. I sent a patch to the mailing list that should improve the check [0].

You can safely ignore the warning for your Debian 12 based container, as it supports the unified cgroup v2 layout.

[0] https://lists.proxmox.com/pipermail/pve-devel/2023-June/057425.html

 

[pipiche]

could it be a similar issue with Fedora 38 based CT ?

the warning:
WARN: Found at least one CT (100) which does not support running in a unified cgroup v2 layout
Consider upgrading the Containers distro or set systemd.unified_cgroup_hierarchy=0 in the Proxmox VE hosts' kernel cmdline! Skipping further CT compat checks.

Checking the systemd version on CT 100

[root@proxy ~]# systemctl --version
systemd 253 (253.7-1.fc38)
+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified

 

[Chris]

could it be a similar issue with Fedora 38 based CT ?

the warning:
WARN: Found at least one CT (100) which does not support running in a unified cgroup v2 layout
Consider upgrading the Containers distro or set systemd.unified_cgroup_hierarchy=0 in the Proxmox VE hosts' kernel cmdline! Skipping further CT compat checks.

Checking the systemd version on CT 100

[root@proxy ~]# systemctl --version
systemd 253 (253.7-1.fc38)
+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified

Hi,
thanks for the report, indeed the way we check for the correct systemd version by identifying the systemd shared object file did not cover Fedoras way of naming that file. I sent a patch which covers this case as well, see https://lists.proxmox.com/pipermail/pve-devel/2023-August/058889.html

 

[aasami]

The same occurs with AlmaLinux 9.2. I have upgraded one of cluster nodes to PVE8 and migrated this LXC to it without any issues.
But when I run again pve7to8 on this upgraded node it still reports that:

WARN: Found at least one CT (30188) which does not support running in a unified cgroup v2 layout
    Consider upgrading the Containers distro or set systemd.unified_cgroup_hierarchy=0 in the Proxmox VE hosts' kernel cmdline! Skipping further CT compat checks.

So AlmaLinux detection should be fixed too.

 

[BigBob]

Same issue here with CentOS 9 Stream (systemd 252). Container was completly new installed in Proxmox 7. How can you check if the script-output is right?

WARN: Found at least one CT (100) which does not support running in a unified cgroup v2 layout
    Consider upgrading the Containers distro or set systemd.unified_cgroup_hierarchy=0 in the Proxmox VE hosts' kernel cmdline! Skipping further CT compat checks.

 

[Chris]

Same issue here with CentOS 9 Stream (systemd 252)

Version 252 of systemd is fine, all above v232 should be compatible (this is used in the pve7to8 check script as compatibility check).

Note that this is a warning, it does not tell you that the container is not compatible as is, just that the script was not able to detect compatibility and you should check the LXC before upgrading your host. The script does not execute any code, it tries to determine the systemd version based on the shared library version in the filename, which is unfortunately very inconsistent in location and filename for different distros.

How can you check if the script-output is right?

You can always check the output of systemctl --version from within the container. This gives you the systemd version including infos about the cgroup hierarchy, e.g default-hierarchy=unified.

 

[Neobin]

Maybe the warning should be rephrased, because this:

WARN: Found at least one CT (100) which does not support running in a unified cgroup v2 layout

does definitely read like a clear incompatibility was detected and not only like:

it does not tell you that the container is not compatible as is, just that the script was not able to detect compatibility and you should check the LXC before upgrading your host.

So, the warning reads, as if the script clearly detected an incompatibility and does not mention anything, that there also might be only a "simple" detection problem.

The info that systemd in the LXC should be / needs to be above v232 should imho also be included in the warning, so that the user directly knows what to check/verify manually.

Just my 2 cents.

 

[Chris]

Maybe the warning should be rephrased, because this:

does definitely read like a clear incompatibility was detected and not only like:


So, the warning reads, as if the script clearly detected an incompatibility and does not mention anything, that there also might be only a "simple" detection problem.

The info that systemd in the LXC should be / needs to be above v232 should imho also be included in the warning, so that the user directly knows what to check/verify manually.

Just my 2 cents.

Hi @Neobin ,
I agree that the text of the warning is rather strict, I assume that this was intentional in order to not be overlooked or ignored. Nevertheless it remains a warning, and a solution is best approached by fixing the detection issues for the different LXCs.

I had a look at both, a AlmaLinux and CentOS 9 Stream container installed from the latest templates available via pveam, but both were detected correctly in my case, with the libsystemd shared object located at /usr/lib/systemd/libsystemd-shared-250.so and am therefore not able to reproduce the issue.

 

[BigBob]

I had a look at both, a AlmaLinux and CentOS 9 Stream container installed from the latest templates available via pveam, but both were detected correctly in my case, with the libsystemd shared object located at /usr/lib/systemd/libsystemd-shared-250.so and am therefore not able to reproduce the issue.

Thanks for your tests.
I tried it myself. A freshly installed CentOS9 from the pve repository correctly has the file. But as soon as I run dnf update the update to systemd-252 takes place and the file disappears. In the directory you can only find:
-rw-r--r-- 1 root root 100 Aug 22 12:44 libsystemd-shared.abignore

 

[Chris]

Thanks for your tests.
I tried it myself. A freshly installed CentOS9 from the pve repository correctly has the file. But as soon as I run dnf update the update to systemd-252 takes place and the file disappears. In the directory you can only find:
-rw-r--r-- 1 root root 100 Aug 22 12:44 libsystemd-shared.abignore

Hi,
I tested with the upgraded version again, and indeed the shared object is now located at /usr/lib64/systemd/libsystemd-shared-252.so. This is however already covered by the patch I sent to fix the Fedora 38 detection, so once the patch is applied and packaged, the detection should work correctly. This is the same for Alma and CentOS 9 Stream.