no web gui, qm list ipcc_send_rec[1] failed: Connection refused

Discussion in 'Proxmox VE: Installation and configuration' started by ds(ds), Feb 8, 2018.

  1. ds(ds)

    ds(ds) Member

    Joined:
    Sep 11, 2011
    Messages:
    60
    Likes Received:
    0
    Suddenly stopped working GUI, virtual machines also do not start

    qm list
    ipcc_send_rec[1] failed: Connection refused
    ipcc_send_rec[2] failed: Connection refused
    ipcc_send_rec[3] failed: Connection refused
    Unable to load access control list: Connection refused


    pveversion -v
    proxmox-ve: 5.1-38 (running kernel: 4.13.13-5-pve)
    pve-manager: 5.1-43 (running version: 5.1-43/bdb08029)
    pve-kernel-4.13.13-2-pve: 4.13.13-33
    pve-kernel-4.13.13-5-pve: 4.13.13-38
    libpve-http-server-perl: 2.0-8
    lvm2: 2.02.168-pve6
    corosync: 2.4.2-pve3
    libqb0: 1.0.1-1
    pve-cluster: 5.0-19
    qemu-server: 5.0-20
    pve-firmware: 2.0-3
    libpve-common-perl: 5.0-25
    libpve-guest-common-perl: 2.0-14
    libpve-access-control: 5.0-7
    libpve-storage-perl: 5.0-17
    pve-libspice-server1: 0.12.8-3
    vncterm: 1.5-3
    pve-docs: 5.1-16
    pve-qemu-kvm: 2.9.1-6
    pve-container: 2.0-18
    pve-firewall: 3.0-5
    pve-ha-manager: 2.0-4
    ksm-control-daemon: 1.2-2
    glusterfs-client: 3.8.8-1
    lxc-pve: 2.1.1-2
    lxcfs: 2.0.8-1
    criu: 2.11.1-1~bpo90
    novnc-pve: 0.6-4
    smartmontools: 6.5+svn4324-1
    zfsutils-linux: 0.7.4-pve2~bpo9


    no cluster
     
  2. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    3,681
    Likes Received:
    338
    what does
    Code:
    systemctl status pve-cluster pveproxy pvedaemon
    
    say?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. ds(ds)

    ds(ds) Member

    Joined:
    Sep 11, 2011
    Messages:
    60
    Likes Received:
    0


    systemctl status pve-cluster pveproxy pvedaemon
    ● pve-cluster.service - The Proxmox VE cluster filesystem
    Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor pres
    Active: failed (Result: exit-code) since Thu 2018-02-08 10:58:28 MSK; 1h 19mi
    Process: 2806 ExecStart=/usr/bin/pmxcfs (code=exited, status=255)
    CPU: 9ms

    Feb 08 10:58:12 pol19-zal systemd[1]: Starting The Proxmox VE cluster filesystem
    Feb 08 10:58:20 pol19-zal pmxcfs[2806]: [database] crit: unable to set WAL mode:
    Feb 08 10:58:20 pol19-zal pmxcfs[2806]: [database] crit: unable to set WAL mode:
    Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] crit: memdb_open failed - unable
    Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] crit: memdb_open failed - unable
    Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] notice: exit proxmox configuratio
    Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Control process exite
    Feb 08 10:58:28 pol19-zal systemd[1]: Failed to start The Proxmox VE cluster fil
    Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Unit entered failed s
    Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Failed with result 'e

    ● pveproxy.service - PVE API Proxy Server
    Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset:
    Active: active (running) since Thu 2018-02-08 10:45:26 MSK; 1h 32min ago
    Process: 1849 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS
    Main PID: 1901 (pveproxy)
    Tasks: 4 (limit: 4915)
    lines 1-23...skipping...
    ● pve-cluster.service - The Proxmox VE cluster filesystem
    Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
    Active: failed (Result: exit-code) since Thu 2018-02-08 10:58:28 MSK; 1h 19min ago
    Process: 2806 ExecStart=/usr/bin/pmxcfs (code=exited, status=255)
    CPU: 9ms

    Feb 08 10:58:12 pol19-zal systemd[1]: Starting The Proxmox VE cluster filesystem...
    Feb 08 10:58:20 pol19-zal pmxcfs[2806]: [database] crit: unable to set WAL mode: disk I/O error#010
    Feb 08 10:58:20 pol19-zal pmxcfs[2806]: [database] crit: unable to set WAL mode: disk I/O error#010
    Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
    Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
    Feb 08 10:58:28 pol19-zal pmxcfs[2806]: [main] notice: exit proxmox configuration filesystem (-1)
    Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Control process exited, code=exited status=255
    Feb 08 10:58:28 pol19-zal systemd[1]: Failed to start The Proxmox VE cluster filesystem.
    Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Unit entered failed state.
    Feb 08 10:58:28 pol19-zal systemd[1]: pve-cluster.service: Failed with result 'exit-code'.

    ● pveproxy.service - PVE API Proxy Server
    Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
    Active: active (running) since Thu 2018-02-08 10:45:26 MSK; 1h 32min ago
    Process: 1849 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
    Main PID: 1901 (pveproxy)
    Tasks: 4 (limit: 4915)
    Memory: 109.0M
    CPU: 57.661s
    CGroup: /system.slice/pveproxy.service
    ├─1901 pveproxy
    ├─7616 pveproxy worker
    ├─7617 pveproxy worker
    └─7618 pveproxy worker

    Feb 08 12:17:57 pol19-zal pveproxy[1901]: worker 7616 started
    Feb 08 12:17:57 pol19-zal pveproxy[1901]: worker 7608 finished
    Feb 08 12:17:57 pol19-zal pveproxy[1901]: starting 1 worker(s)
    Feb 08 12:17:57 pol19-zal pveproxy[1901]: worker 7617 started
    Feb 08 12:17:57 pol19-zal pveproxy[7616]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1642.
    Feb 08 12:17:57 pol19-zal pveproxy[7617]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1642.
    Feb 08 12:17:57 pol19-zal pveproxy[1901]: worker 7609 finished
    Feb 08 12:17:57 pol19-zal pveproxy[1901]: starting 1 worker(s)
    Feb 08 12:17:57 pol19-zal pveproxy[1901]: worker 7618 started
    Feb 08 12:17:57 pol19-zal pveproxy[7618]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1642.

    ● pvedaemon.service - PVE API Daemon
    Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
    Active: active (running) since Thu 2018-02-08 10:45:25 MSK; 1h 32min ago
    Process: 1766 ExecStart=/usr/bin/pvedaemon start (code=exited, status=0/SUCCESS)
    Main PID: 1843 (pvedaemon)
    Tasks: 4 (limit: 4915)
    Memory: 79.7M
    CPU: 680ms
    CGroup: /system.slice/pvedaemon.service
    ├─1843 pvedaemon
    ├─1846 pvedaemon worker
    ├─1847 pvedaemon worker
    └─1848 pvedaemon worker

    Feb 08 10:45:23 pol19-zal systemd[1]: Starting PVE API Daemon...
    Feb 08 10:45:25 pol19-zal pvedaemon[1843]: starting server
    Feb 08 10:45:25 pol19-zal pvedaemon[1843]: starting 3 worker(s)
    Feb 08 10:45:25 pol19-zal pvedaemon[1843]: worker 1846 started
    Feb 08 10:45:25 pol19-zal pvedaemon[1843]: worker 1847 started
    Feb 08 10:45:25 pol19-zal pvedaemon[1843]: worker 1848 started
    Feb 08 10:45:25 pol19-zal systemd[1]: Started PVE API Daemon.
     
  4. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    3,681
    Likes Received:
    338
    it seems that the db /var/lib/pve-cluster/config.db is missing or the disk/fs is broken/damaged
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. ds(ds)

    ds(ds) Member

    Joined:
    Sep 11, 2011
    Messages:
    60
    Likes Received:
    0
    there is no cluster. this is one computer. no NFS too. clean installation 5.
     
  6. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    3,681
    Likes Received:
    338
    that makes no difference, we always use this database even on a single node
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. ds(ds)

    ds(ds) Member

    Joined:
    Sep 11, 2011
    Messages:
    60
    Likes Received:
    0
    how can I recover?
     
  8. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    3,681
    Likes Received:
    338
    i would check your disks first, but then you can delete the file (try to make a copy first) and restore your configs from backup
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice