no web gui, qm list ipcc_send_rec[1] failed: Connection refused

D

ds(ds)

Active Member
Sep 11, 2011
60
0
26
Suddenly stopped working GUI, virtual machines also do not start

qm list
ipcc_send_rec[1] failed: Connection refused
ipcc_send_rec[2] failed: Connection refused
ipcc_send_rec[3] failed: Connection refused
Unable to load access control list: Connection refused


pveversion -v
proxmox-ve: 5.1-38 (running kernel: 4.13.13-5-pve)
pve-manager: 5.1-43 (running version: 5.1-43/bdb08029)
pve-kernel-4.13.13-2-pve: 4.13.13-33
pve-kernel-4.13.13-5-pve: 4.13.13-38
libpve-http-server-perl: 2.0-8
lvm2: 2.02.168-pve6
corosync: 2.4.2-pve3
libqb0: 1.0.1-1
pve-cluster: 5.0-19
qemu-server: 5.0-20
pve-firmware: 2.0-3
libpve-common-perl: 5.0-25
libpve-guest-common-perl: 2.0-14
libpve-access-control: 5.0-7
libpve-storage-perl: 5.0-17
pve-libspice-server1: 0.12.8-3
vncterm: 1.5-3
pve-docs: 5.1-16
pve-qemu-kvm: 2.9.1-6
pve-container: 2.0-18
pve-firewall: 3.0-5
pve-ha-manager: 2.0-4
ksm-control-daemon: 1.2-2
glusterfs-client: 3.8.8-1
lxc-pve: 2.1.1-2
lxcfs: 2.0.8-1
criu: 2.11.1-1~bpo90
novnc-pve: 0.6-4
smartmontools: 6.5+svn4324-1
zfsutils-linux: 0.7.4-pve2~bpo9


no cluster
 
what does
Code: 
systemctl status pve-cluster pveproxy pvedaemon
say?
 
dcsapak said:
what does
Code: 
systemctl status pve-cluster pveproxy pvedaemon
say?
Click to expand...



systemctl status pve-cluster pveproxy pvedaemon
● pve-cluster.service - The Proxmox VE cluster filesystem
Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor pres
Active: failed (Result: exit-code) since Thu 2018-02-08 10:58:28 MSK; 1h 19mi
Process: 2806 ExecStart=/usr/bin/pmxcfs (code=exited, status=255)
CPU: 9ms

Feb 08 10:58:12 pol19-zal systemd[1]: Starting The Proxmox VE cluster filesystem
Feb 08 10:58:20 pol19-zal pmxcfs[2806]: [database] crit: unable to set WAL mode:
Feb 08 10:58:20 pol19-zal pmxcfs[2806]: [database] crit: unable to set WAL mode:
Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] crit: memdb_open failed - unable
Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] crit: memdb_open failed - unable
Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] notice: exit proxmox configuratio
Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Control process exite
Feb 08 10:58:28 pol19-zal systemd[1]: Failed to start The Proxmox VE cluster fil
Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Unit entered failed s
Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Failed with result 'e

● pveproxy.service - PVE API Proxy Server
Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset:
Active: active (running) since Thu 2018-02-08 10:45:26 MSK; 1h 32min ago
Process: 1849 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS
Main PID: 1901 (pveproxy)
Tasks: 4 (limit: 4915)
lines 1-23...skipping...
● pve-cluster.service - The Proxmox VE cluster filesystem
Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2018-02-08 10:58:28 MSK; 1h 19min ago
Process: 2806 ExecStart=/usr/bin/pmxcfs (code=exited, status=255)
CPU: 9ms

Feb 08 10:58:12 pol19-zal systemd[1]: Starting The Proxmox VE cluster filesystem...
Feb 08 10:58:20 pol19-zal pmxcfs[2806]: [database] crit: unable to set WAL mode: disk I/O error#010
Feb 08 10:58:20 pol19-zal pmxcfs[2806]: [database] crit: unable to set WAL mode: disk I/O error#010
Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] notice: exit proxmox configuration filesystem (-1)
Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Control process exited, code=exited status=255
Feb 08 10:58:28 pol19-zal systemd[1]: Failed to start The Proxmox VE cluster filesystem.
Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Unit entered failed state.
Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Failed with result 'exit-code'.

● pveproxy.service - PVE API Proxy Server
Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2018-02-08 10:45:26 MSK; 1h 32min ago
Process: 1849 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
Main PID: 1901 (pveproxy)
Tasks: 4 (limit: 4915)
Memory: 109.0M
CPU: 57.661s
CGroup: /system.slice/pveproxy.service
├─1901 pveproxy
├─7616 pveproxy worker
├─7617 pveproxy worker
└─7618 pveproxy worker

Feb 08 12:17:57 pol19-zal pveproxy[1901]: worker 7616 started
Feb 08 12:17:57 pol19-zal pveproxy[1901]: worker 7608 finished
Feb 08 12:17:57 pol19-zal pveproxy[1901]: starting 1 worker(s)
Feb 08 12:17:57 pol19-zal pveproxy[1901]: worker 7617 started
Feb 08 12:17:57 pol19-zal pveproxy[7616]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1642.
Feb 08 12:17:57 pol19-zal pveproxy[7617]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1642.
Feb 08 12:17:57 pol19-zal pveproxy[1901]: worker 7609 finished
Feb 08 12:17:57 pol19-zal pveproxy[1901]: starting 1 worker(s)
Feb 08 12:17:57 pol19-zal pveproxy[1901]: worker 7618 started
Feb 08 12:17:57 pol19-zal pveproxy[7618]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1642.

● pvedaemon.service - PVE API Daemon
Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2018-02-08 10:45:25 MSK; 1h 32min ago
Process: 1766 ExecStart=/usr/bin/pvedaemon start (code=exited, status=0/SUCCESS)
Main PID: 1843 (pvedaemon)
Tasks: 4 (limit: 4915)
Memory: 79.7M
CPU: 680ms
CGroup: /system.slice/pvedaemon.service
├─1843 pvedaemon
├─1846 pvedaemon worker
├─1847 pvedaemon worker
└─1848 pvedaemon worker

Feb 08 10:45:23 pol19-zal systemd[1]: Starting PVE API Daemon...
Feb 08 10:45:25 pol19-zal pvedaemon[1843]: starting server
Feb 08 10:45:25 pol19-zal pvedaemon[1843]: starting 3 worker(s)
Feb 08 10:45:25 pol19-zal pvedaemon[1843]: worker 1846 started
Feb 08 10:45:25 pol19-zal pvedaemon[1843]: worker 1847 started
Feb 08 10:45:25 pol19-zal pvedaemon[1843]: worker 1848 started
Feb 08 10:45:25 pol19-zal systemd[1]: Started PVE API Daemon.
 
ds(ds) said:
Feb 08 10:58:20 pol19-zal pmxcfs[2806]: [database] crit: unable to set WAL mode: disk I/O error#010
Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
Click to expand...

it seems that the db /var/lib/pve-cluster/config.db is missing or the disk/fs is broken/damaged
 
ds(ds) said:
there is no cluster. this is one computer. no NFS too. clean installation 5.
Click to expand...
that makes no difference, we always use this database even on a single node
 
i would check your disks first, but then you can delete the file (try to make a copy first) and restore your configs from backup
 
I know this is a very old post but I just had the very same problem with pve7 and found that my hostname and /etc/hosts did not match up. Once corrected, pve-cluster launched correctly and all is good. Careful with those hostname edits :-0
 
  • Like
Reactions: sufehmi, JackieBrown and kjkent
amace said:
I know this is a very old post but I just had the very same problem with pve7 and found that my hostname and /etc/hosts did not match up. Once corrected, pve-cluster launched correctly and all is good. Careful with those hostname edits :-0
Click to expand...
This fixed this problem for me as well
 
amace said:
I know this is a very old post but I just had the very same problem with pve7 and found that my hostname and /etc/hosts did not match up. Once corrected, pve-cluster launched correctly and all is good. Careful with those hostname edits :-0
Click to expand...

This just happened to me - thanks amace for the answer.

I was sweating bullets when "pct list" shows error - and /etc/pve/ was empty.
Felt like my heart dropped to the floor.

Thanks again @amace
 
You must log in or register to reply here.
Top Bottom