[SOLVED] upgrade failed - proxmox not working anymore

A

astrakid

Renowned Member
Jun 13, 2013
76
1
73
hi,
i upgraded my proxmox installation. due to some local issues the installation was interrupted and i had to continue later on by dpkg --configure -a and apt fullupgrade.
however, everything seems up to date, but proxmox is not working.
pveproxy is running, but i am not able to reach the gui. access log says error code 501, my browser just displays a blank page after ssl-warning.

::ffff:192.168.1.104 - - [05/09/2019:17:21:43 +0200] "GET / HTTP/1.1" 501 -

pveversion --verbose
root@server:~# pveversion --verbose
proxmox-ve: 6.0-2 (running kernel: 5.0.21-1-pve)
pve-manager: 6.0-7 (running version: 6.0-7/28984024)
pve-kernel-5.0: 6.0-7
pve-kernel-helper: 6.0-7
pve-kernel-4.15: 5.4-8
pve-kernel-5.0.21-1-pve: 5.0.21-2
pve-kernel-4.15.18-20-pve: 4.15.18-46
pve-kernel-4.15.18-19-pve: 4.15.18-45
pve-kernel-4.10.11-1-pve: 4.10.11-9
pve-kernel-4.10.8-1-pve: 4.10.8-7
pve-kernel-4.4.44-1-pve: 4.4.44-84
pve-kernel-4.4.40-1-pve: 4.4.40-82
pve-kernel-4.4.6-1-pve: 4.4.6-48
ceph-fuse: 12.2.11+dfsg1-2.1
corosync: 3.0.2-pve2
criu: 3.11-3
glusterfs-client: 5.5-3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.11-pve1
libpve-access-control: 6.0-2
libpve-apiclient-perl: 3.0-2
libpve-common-perl: 6.0-4
libpve-guest-common-perl: 3.0-1
libpve-http-server-perl: 3.0-2
libpve-storage-perl: 6.0-8
libqb0: 1.0.5-1
lvm2: 2.03.02-pve3
lxc-pve: 3.1.0-64
lxcfs: 3.0.3-pve60
novnc-pve: 1.0.0-60
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.0-7
pve-cluster: 6.0-7
pve-container: 3.0-7
pve-docs: 6.0-4
pve-edk2-firmware: 2.20190614-1
pve-firewall: 4.0-7
pve-firmware: 3.0-2
pve-ha-manager: 3.0-2
pve-i18n: 2.0-3
pve-qemu-kvm: 4.0.0-5
pve-xtermjs: 3.13.2-1
qemu-server: 6.0-7
smartmontools: 7.0-pve2
spiceterm: 3.1-1
vncterm: 1.6-1


any idea to narrow down the issue (and solve it... ;-) )? it is only a homeserver, so it is just me who is affected, but iw ould like to get my VMs running again. :-/


kind regards,
andre
 
Hi,
please check if all the services are up and running correctly systemctl status pve*. Then check the journal journalctl -b
 
Last edited:
hi,

pve-cluster does not start:

Code: 
root@server:/etc/apt# journalctl -xe
--
-- An ExecStart= process belonging to unit pve-cluster.service has exited.
--
-- The process' exit code is 'exited' and its exit status is 255.
Sep 05 17:45:45 server systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- The unit pve-cluster.service has entered the 'failed' state with result 'exit-code'.
Sep 05 17:45:45 server systemd[1]: Failed to start The Proxmox VE cluster filesystem.
-- Subject: A start job for unit pve-cluster.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- A start job for unit pve-cluster.service has finished with a failure.
--
-- The job identifier is 12522 and the job result is failed.
Sep 05 17:45:45 server systemd[1]: Condition check resulted in Corosync Cluster Engine being skipped.
-- Subject: A start job for unit corosync.service has finished successfully
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- A start job for unit corosync.service has finished successfully.
--
-- The job identifier is 12525.
Sep 05 17:45:45 server systemd[1]: pve-cluster.service: Service RestartSec=100ms expired, scheduling restart.
Sep 05 17:45:45 server systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 5.
-- Subject: Automatic restarting of a unit has been scheduled
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Automatic restarting of the unit pve-cluster.service has been scheduled, as the result for
-- the configured Restart= setting for the unit.
Sep 05 17:45:45 server systemd[1]: Stopped The Proxmox VE cluster filesystem.
-- Subject: A stop job for unit pve-cluster.service has finished
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- A stop job for unit pve-cluster.service has finished.
--
-- The job identifier is 12712 and the job result is done.
Sep 05 17:45:45 server systemd[1]: pve-cluster.service: Start request repeated too quickly.
Sep 05 17:45:45 server systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- The unit pve-cluster.service has entered the 'failed' state with result 'exit-code'.
Sep 05 17:45:45 server systemd[1]: Failed to start The Proxmox VE cluster filesystem.
-- Subject: A start job for unit pve-cluster.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- A start job for unit pve-cluster.service has finished with a failure.
--
-- The job identifier is 12712 and the job result is failed.
Sep 05 17:45:45 server systemd[1]: Condition check resulted in Corosync Cluster Engine being skipped.
-- Subject: A start job for unit corosync.service has finished successfully
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- A start job for unit corosync.service has finished successfully.
--
-- The job identifier is 12715.

tried to reinstall pve-cluster, but didn't succeed.

anyway:
Code: 
root@server:/etc/apt# systemctl status pve*
Unit pve-repo-ca-certificates.crt.service could not be found.

how to reinstall that?
 
hmmm... is that missing package related to proxmox-enterprise? i am using the no-subscription version... better said: WAS using... but i want to use it still... ;)
 
journalctl -b errors i think of being related to this issue:
Code: 
Sep 05 17:29:52 server systemd[1]: Failed to start Cgroup management daemon.
[...]
Sep 05 17:30:00 server systemd[1]: Starting Proxmox VE replication runner...
Sep 05 17:30:00 server pvesr[861]: ipcc_send_rec[1] failed: Connection refused
Sep 05 17:30:00 server pvesr[861]: ipcc_send_rec[2] failed: Connection refused
Sep 05 17:30:00 server pvesr[861]: ipcc_send_rec[3] failed: Connection refused
Sep 05 17:30:00 server pvesr[861]: Unable to load access control list: Connection refused
Sep 05 17:30:00 server systemd[1]: pvesr.service: Main process exited, code=exited, status=111/n/a
Sep 05 17:30:00 server systemd[1]: pvesr.service: Failed with result 'exit-code'.
Sep 05 17:30:00 server systemd[1]: Failed to start Proxmox VE replication runner.
[...]
Sep 05 17:30:03 server pmxcfs[1325]: [main] crit: fuse_mount error: File exists
Sep 05 17:30:03 server pmxcfs[1325]: [main] notice: exit proxmox configuration filesystem (-1)
Sep 05 17:30:03 server systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION
Sep 05 17:30:03 server pmxcfs[1325]: [main] crit: fuse_mount error: File exists
Sep 05 17:30:03 server systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Sep 05 17:30:03 server pmxcfs[1325]: [main] notice: exit proxmox configuration filesystem (-1)
Sep 05 17:30:03 server systemd[1]: Failed to start The Proxmox VE cluster filesystem.
 
ok, /etc/pve is an own filesystem - i guess this is my problem. removed the folder, rebooted, and the webgui is back again. only my configuration is gone. any chance to get that back? my VM-files are still there....
 
ok, that information is stored in config.db in /var/lib/pve-cluster. i moved that before, so i restored it and everything seems to be back!!!
thanks a lot for having an ear for me!
 
astrakid said:
ok, that information is stored in config.db in /var/lib/pve-cluster. i moved that before, so i restored it and everything seems to be back!!!
thanks a lot for having an ear for me!
Click to expand...
Glad you are back up and running! Consider performing regular backups to be save ;) (if you do not perform those already).
 
Chris said:
Glad you are back up and running! Consider performing regular backups to be save ;) (if you do not perform those already).
Click to expand...
i am backing up my VMs, that must be enough, because i am not able to backup my single system except a file backup. but i will enhance my existing backups by this important files that i detected now. is that filesystem /etc/pve completely contained in the config.db?

regards,
andre
 
astrakid said:
i am backing up my VMs, that must be enough, because i am not able to backup my single system except a file backup. but i will enhance my existing backups by this important files that i detected now. is that filesystem /etc/pve completely contained in the config.db?

regards,
andre
Click to expand...
Yes, this is a sqlite database which is mounted via fuse to /etc/pve.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom