[SOLVED] issue with greyed out?

killmasta93

Renowned Member
Aug 13, 2017
958
56
68
30
Hi,
Recently having this server only showing grey out vms, It is not in a cluster only single node i checked these services
Rebooting fixes the issue but now its constant not sure why

Thank you

pvecm statusroot@prometheus:~# pvecm status
Corosync config '/etc/pve/corosync.conf' does not exist - is this node part of a cluster?
Cannot initialize CMAP service
root@prometheus:~# service pve-manager status
● pve-guests.service - PVE guests
Loaded: loaded (/lib/systemd/system/pve-guests.service; enabled; vendor preset: enabled)
Active: active (exited) since Mon 2020-05-18 23:13:31 -05; 1 months 3 days ago
Process: 2780 ExecStart=/usr/bin/pvesh --nooutput create /nodes/localhost/startall (code=exited, status=0/SUCCESS)
Main PID: 2780 (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 4915)
Memory: 0B
CPU: 0
CGroup: /system.slice/pve-guests.service

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
root@prometheus:~# service pvestatd status
● pvestatd.service - PVE Status Daemon
Loaded: loaded (/lib/systemd/system/pvestatd.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2020-05-18 23:13:11 -05; 1 months 3 days ago
Process: 2604 ExecStart=/usr/bin/pvestatd start (code=exited, status=0/SUCCESS)
Main PID: 2648 (pvestatd)
Tasks: 1 (limit: 4915)
Memory: 73.2M
CPU: 2h 21min 650ms
CGroup: /system.slice/pvestatd.service
└─2648 pvestatd

Jun 21 19:59:36 prometheus pvestatd[2648]: status update time (8.109 seconds)
Jun 21 20:59:40 prometheus pvestatd[2648]: status update time (51.866 seconds)
Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.

1592795701052.png
 
i checked the system logs the only thing i found was this

Code:
Jun 26 07:16:01 prometheus pvestatd[16740]: start failed - can't acquire lock '/var/run/pvestatd.pid.lock' - Resource temporarily unavailable
Jun 26 07:16:01 prometheus pvestatd[16740]: start failed - can't acquire lock '/var/run/pvestatd.pid.lock' - Resource temporarily unavailable
 
thanks for the reply this is what i get

root@prometheus:~# pveversion -v
proxmox-ve: 5.4-1 (running kernel: 4.15.18-12-pve)
pve-manager: 5.4-3 (running version: 5.4-3/0a6eaa62)
pve-kernel-4.15: 5.3-3
pve-kernel-4.15.18-12-pve: 4.15.18-35
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-8
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-50
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-13
libpve-storage-perl: 5.0-41
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-25
pve-cluster: 5.0-36
pve-container: 2.0-37
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-19
pve-firmware: 2.0-6
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 2.12.1-3
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-50
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2
 
Thanks for the reply, i guess ill have to update it then to see if that error stops as the restart option im getting this

Code:
Jul  6 07:52:04 prometheus pvestatd[14508]: start failed - can't acquire lock '/var/run/pvestatd.pid.lock' - Resource temporarily unavailable
Jul  6 07:52:04 prometheus pvestatd[14508]: start failed - can't acquire lock '/var/run/pvestatd.pid.lock' - Resource temporarily unavailable
 
Hi,

On my side, I have my proxmox greyed out everytime... (once I reboot the server it leasts less than 24h).

When it's greyed out, I have test everything that is in this post.... but nothing is working ....

I have to reboot the server .... trough command line... but often it does not work (failed to talt to init) and I have to reboot "hardly"...

Any idea what is the cause ?
 
and 30 minutes later... greyed out...

I found this in journalctl


Code:
Feb 09 16:42:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Feb 09 16:42:00 proxmox kernel: pvesr[2160]: segfault at 555377289e63 ip 0000555573a4
Feb 09 16:42:00 proxmox kernel: Code: 41 f6 40 22 04 74 47 48 8b 44 24 08 48 83 e8 01
Feb 09 16:42:00 proxmox systemd[1]: pvesr.service: Main process exited, code=killed,
Feb 09 16:42:00 proxmox systemd[1]: pvesr.service: Failed with result 'signal'.
Feb 09 16:42:00 proxmox systemd[1]: Failed to start Proxmox VE replication runner.
Feb 09 16:42:08 proxmox kernel: node[24444]: segfault at 311e804c0526 ip 0000308a6e40
Feb 09 16:42:08 proxmox kernel: Code: 4d 8b 48 07 4d 8b 49 0f 4d 39 4d d8 0f 84 f2 00
Feb 09 16:42:19 proxmox kernel: node[2213]: segfault at 2c7494480010 ip 0000000000cfc
Feb 09 16:42:19 proxmox kernel: Code: 89 f0 83 e0 03 48 83 f8 01 75 da 4d 89 f7 44 89
Feb 09 16:42:22 proxmox kernel: pvestatd[31162]: segfault at 0 ip 00007f31e36b0557 sp
Feb 09 16:42:22 proxmox kernel: Code: fc c3 90 4c 8b 5e ec 48 8b 4e f4 8b 56 fc 4c 89
Feb 09 16:42:22 proxmox systemd[1]: pvestatd.service: Main process exited, code=kille
Feb 09 16:42:22 proxmox systemd[1]: pvestatd.service: Failed with result 'signal'.
Feb 09 16:42:30 proxmox kernel: node[2292]: segfault at 371ed8ec15b5 ip 0000000000d03
Feb 09 16:42:30 proxmox kernel: Code: 01 00 00 4c 39 c6 0f 96 c2 4c 39 c1 48 8b b8 80
Feb 09 16:42:41 proxmox kernel: traps: node[2335] general protection fault ip:f10611
Feb 09 16:42:52 proxmox kernel: node[2374]: segfault at 380bb6f40824 ip 0000000000d03
Feb 09 16:42:52 proxmox kernel: Code: 01 00 00 4c 39 c6 0f 96 c2 4c 39 c1 48 8b b8 80
Feb 09 16:43:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Feb 09 16:43:01 proxmox systemd[1]: pvesr.service: Succeeded.
 
Last edited:
and 30 minutes later... greyed out...

I found this in journalctl


Code:
Feb 09 16:42:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Feb 09 16:42:00 proxmox kernel: pvesr[2160]: segfault at 555377289e63 ip 0000555573a4
Feb 09 16:42:00 proxmox kernel: Code: 41 f6 40 22 04 74 47 48 8b 44 24 08 48 83 e8 01
Feb 09 16:42:00 proxmox systemd[1]: pvesr.service: Main process exited, code=killed,
Feb 09 16:42:00 proxmox systemd[1]: pvesr.service: Failed with result 'signal'.
Feb 09 16:42:00 proxmox systemd[1]: Failed to start Proxmox VE replication runner.
Feb 09 16:42:08 proxmox kernel: node[24444]: segfault at 311e804c0526 ip 0000308a6e40
Feb 09 16:42:08 proxmox kernel: Code: 4d 8b 48 07 4d 8b 49 0f 4d 39 4d d8 0f 84 f2 00
Feb 09 16:42:19 proxmox kernel: node[2213]: segfault at 2c7494480010 ip 0000000000cfc
Feb 09 16:42:19 proxmox kernel: Code: 89 f0 83 e0 03 48 83 f8 01 75 da 4d 89 f7 44 89
Feb 09 16:42:22 proxmox kernel: pvestatd[31162]: segfault at 0 ip 00007f31e36b0557 sp
Feb 09 16:42:22 proxmox kernel: Code: fc c3 90 4c 8b 5e ec 48 8b 4e f4 8b 56 fc 4c 89
Feb 09 16:42:22 proxmox systemd[1]: pvestatd.service: Main process exited, code=kille
Feb 09 16:42:22 proxmox systemd[1]: pvestatd.service: Failed with result 'signal'.
Feb 09 16:42:30 proxmox kernel: node[2292]: segfault at 371ed8ec15b5 ip 0000000000d03
Feb 09 16:42:30 proxmox kernel: Code: 01 00 00 4c 39 c6 0f 96 c2 4c 39 c1 48 8b b8 80
Feb 09 16:42:41 proxmox kernel: traps: node[2335] general protection fault ip:f10611
Feb 09 16:42:52 proxmox kernel: node[2374]: segfault at 380bb6f40824 ip 0000000000d03
Feb 09 16:42:52 proxmox kernel: Code: 01 00 00 4c 39 c6 0f 96 c2 4c 39 c1 48 8b b8 80
Feb 09 16:43:00 proxmox systemd[1]: Starting Proxmox VE replication runner...
Feb 09 16:43:01 proxmox systemd[1]: pvesr.service: Succeeded.
what proxmox version?
 
6.3-2

and today... it is greyed out....
and pvestatd restart does not work...
Failed to start pvestatd.service: Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms)

and when I try to reboot:


root@proxmox:~# reboot
Failed to open initctl fifo: No such device or address
Failed to talk to init daemon.

I found a way to reboot thanks to https://www.svennd.be/systemd-doesnt-want-to-reboot/
systemctl --force --force reboot
 
Last edited:
try the following

Code:
service pvestatd daemon status
   service pvestatd status
  service pve-cluster stop
 service corosync stop
   service pvestatd stop
   service pveproxy stop
   service pvedaemon stop
   service pve-cluster start
   service corosync start
    service pvestatd start
  service pveproxy start
    service pvedaemon start
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!