[SOLVED] After Update: Job for pvestatd.service failed

[anullinger]

Hi, got a Problem after a regular Update with WebIf, after change from No-Sub to Enterprise Repository.
The update run was not completely successful, pve-manager and proxmox-ve are not completly configured.
I tried a apt apgrade run without success, pvestatd.service start is the problem, /etc/pve is empty, This is just a single machine no cluster.

Some ideas where to search?

apt upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
2 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n]
Setting up pve-manager (6.4-8) ...
Job for pvestatd.service failed.
See "systemctl status pvestatd.service" and "journalctl -xe" for details.
dpkg: error processing package pve-manager (--configure):
installed pve-manager package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of proxmox-ve:
proxmox-ve depends on pve-manager; however:
Package pve-manager is not configured yet.

dpkg: error processing package proxmox-ve (--configure):
dependency problems - leaving unconfigured
Errors were encountered while processing:
pve-manager
proxmox-ve
E: Sub-process /usr/bin/dpkg returned an error code (1)

----
root@prox-ro-02:~# systemctl status pvestatd.service
● pvestatd.service - PVE Status Daemon
Loaded: loaded (/lib/systemd/system/pvestatd.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2021-04-08 14:55:14 CEST; 2 months 9 days ago
Process: 31432 ExecReload=/usr/bin/pvestatd restart (code=exited, status=111)
Main PID: 1981 (pvestatd)
Tasks: 1 (limit: 4915)
Memory: 112.0M
CGroup: /system.slice/pvestatd.service
└─1981 pvestatd

Jun 18 09:22:34 prox-ro-02 pvestatd[1981]: ipcc_send_rec[1] failed: Connection refused
Jun 18 09:22:34 prox-ro-02 pvestatd[1981]: ipcc_send_rec[2] failed: Connection refused
Jun 18 09:22:34 prox-ro-02 pvestatd[1981]: ipcc_send_rec[3] failed: Connection refused
Jun 18 09:22:34 prox-ro-02 pvestatd[1981]: ipcc_send_rec[4] failed: Connection refused
Jun 18 09:22:34 prox-ro-02 pvestatd[1981]: status update error: Connection refused

-----
proxmox-ve: not correctly installed (running kernel: 5.4.73-1-pve)
pve-manager: not correctly installed (running version: 6.4-8/185e14db)
pve-kernel-5.4: 6.4-2
pve-kernel-helper: 6.4-2
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.0.3-1
libpve-access-control: 6.4-1
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-3
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-3
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.8-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.5-6
pve-cluster: 6.4-1
pve-container: 3.3-5
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.2-4
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.4-pve1

 

[anullinger]

i have not tested a reboot yet, i am currently only remote on the machine and it is far away

 

[anullinger]

The /etc/pve Folder is empty, so i tried to manualy start pve-cluster service. Failed, too. But I have an entry that says more.

- The job identifier is 5637885.
Jun 18 10:13:38 prox-ro-02 pmxcfs[8779]: [database] crit: found entry with duplicate name 'qemu-server' - Ainode = 0x000000000001F8D6, parent = 0x0000
Jun 18 10:13:38 prox-ro-02 pmxcfs[8779]: [database] crit: DB load failed
Jun 18 10:13:38 prox-ro-02 pmxcfs[8779]: [database] crit: found entry with duplicate name 'qemu-server' - Ainode = 0x000000000001F8D6, parent = 0x0000
Jun 18 10:13:38 prox-ro-02 pmxcfs[8779]: [database] crit: DB load failed
Jun 18 10:13:38 prox-ro-02 pmxcfs[8779]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
Jun 18 10:13:38 prox-ro-02 pmxcfs[8779]: [main] notice: exit proxmox configuration filesystem (-1)
Jun 18 10:13:38 prox-ro-02 pmxcfs[8779]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
Jun 18 10:13:38 prox-ro-02 pmxcfs[8779]: [main] notice: exit proxmox configuration filesystem (-1)
Jun 18 10:13:38 prox-ro-02 systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION

 

[anullinger]

Ok got an Dublicate Entry in /var/lib/pve-cluster/config.db

sqlite3 config.db
sqlite> select * from tree where name='qemu-server';
129238|129237|129238|0|1617886514|4|qemu-server|
129407|129237|129407|0|1617886741|4|qemu-server|

i made a backup copy to tamper with
Shoud i simply delete an entry? Which one? How to choose the right Inode?

 

[anullinger]

Ok found a solution: https://forum.proxmox.com/threads/pveproxy-dev-fuse-not-mounted-to-etc-pve.34968/#post-368567
I deleted the older entry