[SOLVED] server failure after upgrade

justingr

New Member
Dec 26, 2022
7
0
1
good day everybody,
I have a problem with a proxmox server after upgrade.
it came after reboot with hostname CloudLinux8-CHANGE-CLONE and practically no VEs and nothing in /etc/pve
the server looks OK in terms of hardware, I see all shell entries in history and the upgrade info in /var/log/messages but nothing seems to work...
any hints on debugging this issue?

I've noticed /etc/hosts was changed during upgrade. I"ve restored /etc/hosts from backups but server is not coming up, stuck in a kind of boot loop...

thanks,
Justin
 
Last edited:
server is down now and I'm on my way to the DC.
tech guy that did the upgrade said he ran
apt update, apt upgrade and the reboot
on another machine it worked well and the machine came back online with new kernel..

thanks,
Justin
 
Hi,

We highly recommend using apt dist-upgrade instead of apt upgrade

I would check the syslog/journalctl in order to see if there is an error message, and please provide us with the output of pveversion -v command.
 
hi,

hosts file is getting overwritten with the following:
127.0.1.1 CloudLinux8-CHANGE-CLONE CloudLinux8-CHANGE-CLONE
127.0.0.1 localhost

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts


here's the history.log :
Start-Date: 2022-12-26 16:40:50
Commandline: apt upgrade
Install: proxmox-mail-forward:amd64 (0.1.1-1, automatic), pve-kernel-5.15.83-1-pve:amd64 (5.15.83-1, automatic), proxmox-offline-mirror-helper:amd64 (0.5.0-1, automatic), proxmox-offline-mirror-docs:amd64 (0.5.0-1, automatic)
Upgrade: dpkg:amd64 (1.20.10, 1.20.12), libntfs-3g883:amd64 (1:2017.3.23AR.3-4+deb11u2, 1:2017.3.23AR.3-4+deb11u3), tcpdump:amd64 (4.99.0-2, 4.99.0-2+deb11u1), pve-docs:amd64 (7.2-2, 7.3-1), libcurl4:amd64 (7.74.0-1.3+deb11u1, 7.74.0-1.3+deb11u3), libc6-i386:amd64 (2.31-13+deb11u3, 2.31-13+deb11u5), udev:amd64 (247.3-7, 247.3-7+deb11u1), krb5-locales:amd64 (1.18.3-6+deb11u1, 1.18.3-6+deb11u3), libext2fs2:amd64 (1.46.2-2, 1.46.5-2~bpo11+2), bind9-host:amd64 (1:9.16.27-1~deb11u1, 1:9.16.33-1~deb11u1), libgssapi-krb5-2:amd64 (1.18.3-6+deb11u1, 1.18.3-6+deb11u3), libgssapi-krb5-2:i386 (1.18.3-6+deb11u1, 1.18.3-6+deb11u3), libcurl3-gnutls:amd64 (7.74.0-1.3+deb11u1, 7.74.0-1.3+deb11u3), openssh-client:amd64 (1:8.4p1-5, 1:8.4p1-5+deb11u1), proxmox-widget-toolkit:amd64 (3.5.1, 3.5.3), libpve-rs-perl:amd64 (0.6.1, 0.7.3), corosync:amd64 (3.1.5-pve2, 3.1.7-pve1), libintl-perl:amd64 (1.26-3, 1.26-3+deb11u1), libnftables1:amd64 (0.9.8-3.1, 0.9.8-3.1+deb11u1), pve-firmware:amd64 (3.4-2, 3.6-2), gpg:amd64 (2.2.27-2+deb11u1, 2.2.27-2+deb11u2), libtirpc-common:amd64 (1.3.1-1, 1.3.1-1+deb11u1), tzdata:amd64 (2021a-1+deb11u4, 2021a-1+deb11u8), zfs-zed:amd64 (2.1.4-pve1, 2.1.7-pve1), libldb2:amd64 (2:2.2.3-2~deb11u1, 2:2.2.3-2~deb11u2), ntfs-3g:amd64 (1:2017.3.23AR.3-4+deb11u2, 1:2017.3.23AR.3-4+deb11u3), libtasn1-6:amd64 (4.16.0-2, 4.16.0-2+deb11u1), libpam-systemd:amd64 (247.3-7, 247.3-7+deb11u1), zfs-initramfs:amd64 (2.1.4-pve1, 2.1.7-pve1), spl:amd64 (2.1.4-pve1, 2.1.7-pve1), ssh:amd64 (1:8.4p1-5, 1:8.4p1-5+deb11u1), pve-qemu-kvm:amd64 (6.2.0-10, 7.1.0-4), libnvpair3linux:amd64 (2.1.4-pve1, 2.1.7-pve1), xsltproc:amd64 (1.1.34-4, 1.1.34-4+deb11u1), libavahi-common-data:amd64 (0.8-5, 0.8-5+deb11u1), libproxmox-acme-perl:amd64 (1.4.2, 1.4.3), libpve-cluster-api-perl:amd64 (7.2-1, 7.3-1), pve-ha-manager:amd64 (3.3-4, 3.5.1), libexpat1:amd64 (2.2.10-2+deb11u3, 2.2.10-2+deb11u5), grub-pc-bin:amd64 (2.04-20, 2.06-3~deb11u5), procmail:amd64 (3.22-26, 3.22-26+deb11u1), swtpm-libs:amd64 (0.7.1~bpo11+1, 0.8.0~bpo11+2), pve-lxc-syscalld:amd64 (1.1.1-1, 1.2.2-1), swtpm-tools:amd64 (0.7.1~bpo11+1, 0.8.0~bpo11+2), libuutil3linux:amd64 (2.1.4-pve1, 2.1.7-pve1), libfribidi0:amd64 (1.0.8-2, 1.0.8-2+deb11u1), libpve-storage-perl:amd64 (7.2-5, 7.3-1), libpixman-1-0:amd64 (0.40.0-1, 0.40.0-1.1~deb11u1), libsystemd0:amd64 (247.3-7, 247.3-7+deb11u1), libfreetype6:amd64 (2.10.4+dfsg-1, 2.10.4+dfsg-1+deb11u1), libzpool5linux:amd64 (2.1.4-pve1, 2.1.7-pve1), libnss-systemd:amd64 (247.3-7, 247.3-7+deb11u1), libpve-guest-common-perl:amd64 (4.1-2, 4.2-3), logrotate:amd64 (3.18.0-2, 3.18.0-2+deb11u1), libwbclient0:amd64 (2:4.13.13+dfsg-1~deb11u3, 2:4.13.13+dfsg-1~deb11u5), openssh-server:amd64 (1:8.4p1-5, 1:8.4p1-5+deb11u1), libdbus-1-3:amd64 (1.12.20-2, 1.12.24-0+deb11u1), libvotequorum8:amd64 (3.1.5-pve2, 3.1.7-pve1), gnupg:amd64 (2.2.27-2+deb11u1, 2.2.27-2+deb11u2), gpg-wks-server:amd64 (2.2.27-2+deb11u1, 2.2.27-2+deb11u2), libkrb5support0:amd64 (1.18.3-6+deb11u1, 1.18.3-6+deb11u3), libkrb5support0:i386 (1.18.3-6+deb11u1, 1.18.3-6+deb11u3), libquorum5:amd64 (3.1.5-pve2, 3.1.7-pve1), libsmbclient:amd64 (2:4.13.13+dfsg-1~deb11u3, 2:4.13.13+dfsg-1~deb11u5), swtpm:amd64 (0.7.1~bpo11+1, 0.8.0~bpo11+2), libxml2:amd64 (2.9.10+dfsg-6.7+deb11u2, 2.9.10+dfsg-6.7+deb11u3), pve-cluster:amd64 (7.2-1, 7.3-1), systemd:amd64 (247.3-7, 247.3-7+deb11u1), libproxmox-rs-perl:amd64 (0.1.1, 0.2.1), libudev1:amd64 (247.3-7, 247.3-7+deb11u1), gpg-agent:amd64 (2.2.27-2+deb11u1, 2.2.27-2+deb11u2), isc-dhcp-common:amd64 (4.4.1-2.3, 4.4.1-2.3+deb11u1), proxmox-ve:amd64 (7.2-1, 7.3-1), lxc-pve:amd64 (4.0.12-1, 5.0.0-3), libcmap4:amd64 (3.1.5-pve2, 3.1.7-pve1), proxmox-backup-file-restore:amd64 (2.2.3-1, 2.3.1-1), libc6:amd64 (2.31-13+deb11u3, 2.31-13+deb11u5), libc6:i386 (2.31-13+deb11u3, 2.31-13+deb11u5), locales:amd64 (2.31-13+deb11u3, 2.31-13+deb11u5), libcfg7:amd64 (3.1.5-pve2, 3.1.7-pve1), debootstrap:amd64 (1.0.123, 1.0.123+deb11u1), libkrb5-3:amd64 (1.18.3-6+deb11u1, 1.18.3-6+deb11u3), libkrb5-3:i386 (1.18.3-6+deb11u1, 1.18.3-6+deb11u3), qemu-server:amd64 (7.2-3, 7.3-2), gpgv:amd64 (2.2.27-2+deb11u1, 2.2.27-2+deb11u2), libpve-access-control:amd64 (7.2-2, 7.3-1), bash:amd64 (5.1-2+b3, 5.1-2+deb11u1), pve-container:amd64 (4.2-1, 4.4-2), isc-dhcp-client:amd64 (4.4.1-2.3, 4.4.1-2.3+deb11u1), libproxmox-acme-plugins:amd64 (1.4.2, 1.4.3), libcpg4:amd64 (3.1.5-pve2, 3.1.7-pve1), pve-i18n:amd64 (2.7-2, 2.8-1), bind9-dnsutils:amd64 (1:9.16.27-1~deb11u1, 1:9.16.33-1~deb11u1), base-files:amd64 (11.1+deb11u3, 11.1+deb11u6), libk5crypto3:amd64 (1.18.3-6+deb11u1, 1.18.3-6+deb11u3), libk5crypto3:i386 (1.18.3-6+deb11u1, 1.18.3-6+deb11u3), proxmox-archive-keyring:amd64 (2.0, 2.2), libtpms0:amd64 (0.9.2~bpo11+1, 0.9.5~bpo11+1), gnutls-bin:amd64 (3.7.1-5, 3.7.1-5+deb11u2), gpgsm:amd64 (2.2.27-2+deb11u1, 2.2.27-2+deb11u2), libc6-dev-x32:amd64 (2.31-13+deb11u3, 2.31-13+deb11u5), libxslt1.1:amd64 (1.1.34-4, 1.1.34-4+deb11u1), proxmox-backup-client:amd64 (2.2.3-1, 2.3.1-1), distro-info-data:amd64 (0.51+deb11u1, 0.51+deb11u3), python3-ldb:amd64 (2:2.2.3-2~deb11u1, 2:2.2.3-2~deb11u2), libpcre2-8-0:amd64 (10.36-2, 10.36-2+deb11u1), libavahi-common3:amd64 (0.8-5, 0.8-5+deb11u1), grub-efi-amd64-bin:amd64 (2.04-20, 2.06-3~deb11u5), grub2-common:amd64 (2.04-20, 2.06-3~deb11u5), smbclient:amd64 (2:4.13.13+dfsg-1~deb11u3, 2:4.13.13+dfsg-1~deb11u5), libpve-http-server-perl:amd64 (4.1-2, 4.1-5), libc-dev-bin:amd64 (2.31-13+deb11u3, 2.31-13+deb11u5), libssl1.1:amd64 (1.1.1n-0+deb11u2, 1.1.1n-0+deb11u3), libssl1.1:i386 (1.1.1n-0+deb11u2, 1.1.1n-0+deb11u3), pve-manager:amd64 (7.2-5, 7.3-4), libpve-common-perl:amd64 (7.2-2, 7.3-1), openssh-sftp-server:amd64 (1:8.4p1-5, 1:8.4p1-5+deb11u1), nano:amd64 (5.4-2, 5.4-2+deb11u2), libc-l10n:amd64 (2.31-13+deb11u3, 2.31-13+deb11u5), grub-common:amd64 (2.04-20, 2.06-3~deb11u5), bind9-libs:amd64 (1:9.16.27-1~deb11u1, 1:9.16.33-1~deb11u1), samba-libs:amd64 (2:4.13.13+dfsg-1~deb11u3, 2:4.13.13+dfsg-1~deb11u5), libc-bin:amd64 (2.31-13+deb11u3, 2.31-13+deb11u5), libc-devtools:amd64 (2.31-13+deb11u3, 2.31-13+deb11u5), librados2-perl:amd64 (1.2-1, 1.3-1), pve-kernel-5.15:amd64 (7.2-5, 7.3-1), libc6-dev:amd64 (2.31-13+deb11u3, 2.31-13+deb11u5), libc6-dev:i386 (2.31-13+deb11u3, 2.31-13+deb11u5), dirmngr:amd64 (2.2.27-2+deb11u1, 2.2.27-2+deb11u2), libzfs4linux:amd64 (2.1.4-pve1, 2.1.7-pve1), libksba8:amd64 (1.5.0-3, 1.5.0-3+deb11u2), libc6-dev-i386:amd64 (2.31-13+deb11u3, 2.31-13+deb11u5), systemd-sysv:amd64 (247.3-7, 247.3-7+deb11u1), samba-common:amd64 (2:4.13.13+dfsg-1~deb11u3, 2:4.13.13+dfsg-1~deb11u5), gnupg-utils:amd64 (2.2.27-2+deb11u1, 2.2.27-2+deb11u2), libgnutls30:amd64 (3.7.1-5, 3.7.1-5+deb11u2), curl:amd64 (7.74.0-1.3+deb11u1, 7.74.0-1.3+deb11u3), logsave:amd64 (1.46.2-2, 1.46.5-2~bpo11+2), libvirglrenderer1:amd64 (0.8.2-5, 0.8.2-5+deb11u1), pve-firewall:amd64 (4.2-5, 4.2-7), gnupg-l10n:amd64 (2.2.27-2+deb11u1, 2.2.27-2+deb11u2), gpg-wks-client:amd64 (2.2.27-2+deb11u1, 2.2.27-2+deb11u2), libcorosync-common4:amd64 (3.1.5-pve2, 3.1.7-pve1), libnozzle1:amd64 (1.24-pve1, 1.24-pve2), zlib1g:amd64 (1:1.2.11.dfsg-2+deb11u1, 1:1.2.11.dfsg-2+deb11u2), libintl-xs-perl:amd64 (1.26-3, 1.26-3+deb11u1), libc6-x32:amd64 (2.31-13+deb11u3, 2.31-13+deb11u5), libtirpc3:amd64 (1.3.1-1, 1.3.1-1+deb11u1), libtirpc3:i386 (1.3.1-1, 1.3.1-1+deb11u1), dbus:amd64 (1.12.20-2, 1.12.24-0+deb11u1), libgnutlsxx28:amd64 (3.7.1-5, 3.7.1-5+deb11u2), libknet1:amd64 (1.24-pve1, 1.24-pve2), grub-pc:amd64 (2.04-20, 2.06-3~deb11u5), libavahi-client3:amd64 (0.8-5, 0.8-5+deb11u1), gpgconf:amd64 (2.2.27-2+deb11u1, 2.2.27-2+deb11u2), libgnutls-dane0:amd64 (3.7.1-5, 3.7.1-5+deb11u2), libss2:amd64 (1.46.2-2, 1.46.5-2~bpo11+2), libtirpc-dev:amd64 (1.3.1-1, 1.3.1-1+deb11u1), libtirpc-dev:i386 (1.3.1-1, 1.3.1-1+deb11u1), pve-edk2-firmware:amd64 (3.20210831-2, 3.20220526-1), pve-kernel-helper:amd64 (7.2-5, 7.3-1), zfsutils-linux:amd64 (2.1.4-pve1, 2.1.7-pve1), postfix:amd64 (3.5.6-1+b1, 3.5.17-0+deb11u1), openssl:amd64 (1.1.1n-0+deb11u2, 1.1.1n-0+deb11u3), e2fsprogs:amd64 (1.46.2-2, 1.46.5-2~bpo11+2), libhttp-daemon-perl:amd64 (6.12-1, 6.12-1+deb11u1), libpve-cluster-perl:amd64 (7.2-1, 7.3-1), nftables:amd64 (0.9.8-3.1, 0.9.8-3.1+deb11u1), linux-libc-dev:amd64 (5.10.120-1, 5.10.158-2), linux-libc-dev:i386 (5.10.120-1, 5.10.158-2)
End-Date: 2022-12-26 16:43:34

Start-Date: 2022-12-26 20:19:09
Commandline: apt reinstall proxmox-ve
Reinstall: proxmox-ve:amd64 (7.3-1)
End-Date: 2022-12-26 20:19:09

Start-Date: 2022-12-26 20:30:43
Commandline: apt install apt-file
Install: libexporter-tiny-perl:amd64 (1.002002-1, automatic), liblist-moreutils-perl:amd64 (0.430-2, automatic), apt-file:amd64 (3.2.2), libregexp-assemble-perl:amd64 (0.36-1.1, automatic), liblist-moreutils-xs-perl:amd64 (0.430-2, automatic)
End-Date: 2022-12-26 20:30:44





root@CloudLinux8-CHANGE-CLONE:~# pveversion -v
proxmox-ve: 7.3-1 (running kernel: 5.15.83-1-pve)
pve-manager: 7.3-4 (running version: 7.3-4/d69b70d4)
pve-kernel-5.15: 7.3-1
pve-kernel-helper: 7.3-1
pve-kernel-5.13: 7.1-9
pve-kernel-5.15.83-1-pve: 5.15.83-1
pve-kernel-5.15.35-3-pve: 5.15.35-6
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-2-pve: 5.13.19-4
ceph-fuse: 15.2.15-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.3
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.3-1
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.3-1
libpve-guest-common-perl: 4.2-3
libpve-http-server-perl: 4.1-5
libpve-storage-perl: 7.3-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.3.1-1
proxmox-backup-file-restore: 2.3.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.0-1
proxmox-widget-toolkit: 3.5.3
pve-cluster: 7.3-1
pve-container: 4.4-2
pve-docs: 7.3-1
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-7
pve-firmware: 3.6-2
pve-ha-manager: 3.5.1
pve-i18n: 2.8-1
pve-qemu-kvm: 7.1.0-4
pve-xtermjs: 4.16.0-1
qemu-server: 7.3-2
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+2
vncterm: 1.7-1
zfsutils-linux: 2.1.7-pve1


any help restoring the server to operation is appreciated :)

thanks,
Justin
 
journalctl log:

Dec 27 13:29:18 CloudLinux8-CHANGE-CLONE pveproxy[1925]: worker 2129 finished
Dec 27 13:29:18 CloudLinux8-CHANGE-CLONE pveproxy[1925]: worker 2131 started
Dec 27 13:29:18 CloudLinux8-CHANGE-CLONE pveproxy[2130]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1899.
Dec 27 13:29:18 CloudLinux8-CHANGE-CLONE pveproxy[2131]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1899.
Dec 27 13:29:23 CloudLinux8-CHANGE-CLONE pveproxy[1925]: starting 1 worker(s)
Dec 27 13:29:23 CloudLinux8-CHANGE-CLONE pveproxy[1925]: worker 2135 started
Dec 27 13:29:23 CloudLinux8-CHANGE-CLONE pveproxy[2130]: worker exit
Dec 27 13:29:23 CloudLinux8-CHANGE-CLONE pveproxy[2131]: worker exit
Dec 27 13:29:23 CloudLinux8-CHANGE-CLONE pveproxy[2135]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1899.
Dec 27 13:29:23 CloudLinux8-CHANGE-CLONE pveproxy[1925]: worker 2130 finished
Dec 27 13:29:23 CloudLinux8-CHANGE-CLONE pveproxy[1925]: starting 1 worker(s)
Dec 27 13:29:23 CloudLinux8-CHANGE-CLONE pveproxy[1925]: worker 2136 started
Dec 27 13:29:24 CloudLinux8-CHANGE-CLONE pveproxy[1925]: worker 2131 finished
Dec 27 13:29:24 CloudLinux8-CHANGE-CLONE pveproxy[1925]: starting 1 worker(s)
Dec 27 13:29:24 CloudLinux8-CHANGE-CLONE pveproxy[1925]: worker 2137 started
Dec 27 13:29:24 CloudLinux8-CHANGE-CLONE pveproxy[2136]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1899.
Dec 27 13:29:24 CloudLinux8-CHANGE-CLONE pveproxy[2137]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1899.
~
~
 
Can you please try to do apt update && apt dist-upgrade command? If this command didn't help, please provide us with the following command from (CloudLinux8-CHANGE-CLONE) node

Bash:
cat /etc/hosts
journalctl -b -u pve-cluster.service
ls -l /etc/pve/local/pve-ssl*
 
Did you by any chance install the: cloud-init package directly on the PVE-host?

PS.: Please use code-tags for output, otherwise it is terrible to read.
 
  • Like
Reactions: Moayad
hi,

root@CloudLinux8-CHANGE-CLONE:~# apt update
Hit:1 http://security.debian.org bullseye-security InRelease
Hit:2 http://ftp.debian.org/debian bullseye InRelease
Get:3 http://ftp.debian.org/debian bullseye-updates InRelease [44.1 kB]
Hit:4 http://download.proxmox.com/debian/pve bullseye InRelease
Fetched 44.1 kB in 3s (17.6 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
1 package can be upgraded. Run 'apt list --upgradable' to see it.
root@CloudLinux8-CHANGE-CLONE:~# apt dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
libcom-err2
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.



--------------------------------------------



root@CloudLinux8-CHANGE-CLONE:~# cat /etc/hosts
# Your system has configured 'manage_etc_hosts' as True.
# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.debian.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
# /etc/cloud/cloud.cfg or cloud-config from user-data
#
127.0.1.1 CloudLinux8-CHANGE-CLONE CloudLinux8-CHANGE-CLONE
127.0.0.1 localhost

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts



--------------------------------------------


root@CloudLinux8-CHANGE-CLONE:~# journalctl -b -u pve-cluster.service
-- Journal begins at Fri 2022-02-11 15:35:33 IST, ends at Tue 2022-12-27 13:49:22 IST. --
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: Starting The Proxmox VE cluster filesystem...
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE pmxcfs[1905]: [main] crit: Unable to get local IP address
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE pmxcfs[1905]: [main] crit: Unable to get local IP address
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: Failed to start The Proxmox VE cluster filesystem.
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 1.
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: Stopped The Proxmox VE cluster filesystem.
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: Starting The Proxmox VE cluster filesystem...
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE pmxcfs[1913]: [main] crit: Unable to get local IP address
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE pmxcfs[1913]: [main] crit: Unable to get local IP address
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: Failed to start The Proxmox VE cluster filesystem.
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 2.
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: Stopped The Proxmox VE cluster filesystem.
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: Starting The Proxmox VE cluster filesystem...
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE pmxcfs[1914]: [main] crit: Unable to get local IP address
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE pmxcfs[1914]: [main] crit: Unable to get local IP address
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Dec 27 13:47:29 CloudLinux8-CHANGE-CLONE systemd[1]: Failed to start The Proxmox VE cluster filesystem.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 3.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: Stopped The Proxmox VE cluster filesystem.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: Starting The Proxmox VE cluster filesystem...
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE pmxcfs[1915]: [main] crit: Unable to get local IP address
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE pmxcfs[1915]: [main] crit: Unable to get local IP address
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: Failed to start The Proxmox VE cluster filesystem.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 4.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: Stopped The Proxmox VE cluster filesystem.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: Starting The Proxmox VE cluster filesystem...
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE pmxcfs[1920]: [main] crit: Unable to get local IP address
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE pmxcfs[1920]: [main] crit: Unable to get local IP address
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: Failed to start The Proxmox VE cluster filesystem.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 5.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: Stopped The Proxmox VE cluster filesystem.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Start request repeated too quickly.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Dec 27 13:47:30 CloudLinux8-CHANGE-CLONE systemd[1]: Failed to start The Proxmox VE cluster filesystem.



-----------------------------


ls -l /etc/pve/local/pve-ssl*
ls: cannot access '/etc/pve/local/pve-ssl*': No such file or directory
 
Hello,

Thank you for the output!

As @Neobin mentioned, the cloud-init maybe somehow installed on the PVE host, which explains why the /etc/hosts file gets reset at reboot, I would remove the cloud-init from the PVE host and fix the below files:

Code:
- /etc/hosts
- /etc/hostname
- /etc/network/interfaces

Then you should reboot the PVE host.

Hopefully, that helps.
 
ok, server is back online after removing cloud-init and restoring the above files.
many thanks for the lp, much appreciated !

Justin
 
Glad to read that you fix the issue yourself!

I will go ahead and set your thread as [SOLVED] in order to help people who have the same issue.
 
Hi Team,
Thanks a lot for the help, the above have resolved my problem too with the proxmox.
Many thanks and have great day.
 
  • Like
Reactions: Moayad