Panic in cluster

[ufm]

Hi!

I received such a message in the log on all servers in the cluster:

Apr 21 06:11:57 pve07 pmxcfs[3704]: [status] notice: received log
Apr 21 06:12:00 pve07 pmxcfs[3704]: [status] notice: members: 1/2428, 2/3487, 3/3704, 4/3686
Apr 21 06:12:00 pve07 pmxcfs[3704]: [status] notice: starting data syncronisation
Apr 21 06:12:00 pve07 pmxcfs[3704]: [dcdb] notice: members: 1/2428, 2/3487, 3/3704, 4/3686
Apr 21 06:12:00 pve07 pmxcfs[3704]: [dcdb] notice: starting data syncronisation
Apr 21 06:12:00 pve07 pmxcfs[3704]: [dcdb] notice: received sync request (epoch 1/2428/00000012)
Apr 21 06:12:00 pve07 pmxcfs[3704]: [status] notice: received sync request (epoch 1/2428/00000012)
Apr 21 06:12:00 pve07 kernel: [978964.570793] cfs_loop[3709]: segfault at 7fc2edc4432c ip 0000564b1b0486b0 sp 00007fc28e2333b8 error 4 in pmxcfs[5
64b1b02b000+2b000]
Apr 21 06:12:00 pve07 systemd[1]: pve-cluster.service: Main process exited, code=killed, status=11/SEGV
Apr 21 06:12:00 pve07 systemd[1]: pve-cluster.service: Unit entered failed state.
Apr 21 06:12:00 pve07 systemd[1]: pve-cluster.service: Failed with result 'signal'.
Apr 21 06:12:00 pve07 systemd[1]: Starting Proxmox VE replication runner...
Apr 21 06:12:00 pve07 pve-ha-crm[4138]: lost lock 'ha_manager_lock - can't create '/etc/pve/priv/lock' (pmxcfs not mounted?)
Apr 21 06:12:01 pve07 pvesr[14066]: ipcc_send_rec[1] failed: Connection refused
Apr 21 06:12:01 pve07 pvesr[14066]: ipcc_send_rec[2] failed: Connection refused
Apr 21 06:12:01 pve07 pvesr[14066]: ipcc_send_rec[3] failed: Connection refused
[...]
Apr 21 06:13:04 pve07 pvestatd[4027]: ipcc_send_rec[1] failed: Connection refused
Apr 21 06:13:04 pve07 pvestatd[4027]: ipcc_send_rec[2] failed: Connection refused
Apr 21 06:13:04 pve07 pvestatd[4027]: ipcc_send_rec[3] failed: Connection refused
Apr 21 06:13:04 pve07 pvestatd[4027]: ipcc_send_rec[4] failed: Connection refused
Apr 21 06:13:04 pve07 pvestatd[4027]: status update error: Connection refused
Apr 21 06:14:33 pve07 systemd-modules-load[1141]: Inserted module 'iscsi_tcp'
Apr 21 06:14:33 pve07 kernel: [    0.000000] Linux version 4.15.18-12-pve (build@pve) (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)) #1 SMP PVE 4.15.18-35 (Wed, 13 Mar 2019 08:24:42 +0100) ()

I.E. all nodes at 06:12 segfaulted in kernal and 06:14 rebooted.

Version: pve-manager/5.3-12/5fbbbaf6 (running kernel: 4.15.18-12-pve)

What could it be and how to get rid of it?

 

[mira]

This might be related to https://forum.proxmox.com/threads/segfault-on-all-cluster-nodes.51960/

 

[ufm]

As I understand - this "floating bug" still exist?

 

[mira]

As far as I can see it still exists. We could not yet pinpoint the issue.