No host metadata after adding NFS datastore

prox091

New Member
Aug 29, 2012
1
0
1
Hi,

I have three nodes. After adding an NFS datastore (FreeNAS), the status and metadata of the second node is wrong. The node is shown as offline and I got no metadata about the VM's. I only see the VM id's, but not their status (running/stopped) and other information (disk, network, ...).

I get the management interface from the second node, but I can't login. After waiting 30 seconds I got: "Login failed. Please try again.".

From the other nodes, login succeeds. Rarely, I also see that node 1 or 2 is shown as offline.

pveversion -v:
Code:
pve-manager: 2.1-1 (pve-manager/2.1/f9b0f63a)
running kernel: 2.6.32-12-pve
proxmox-ve-2.6.32: 2.1-68
pve-kernel-2.6.32-11-pve: 2.6.32-66
pve-kernel-2.6.32-12-pve: 2.6.32-68
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.3-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-2
pve-cluster: 1.0-26
qemu-server: 2.0-39
pve-firmware: 1.0-16
libpve-common-perl: 1.0-27
libpve-access-control: 1.0-21
libpve-storage-perl: 2.0-18
vncterm: 1.0-2
vzctl: 3.0.30-2pve5
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-9
ksm-control-daemon: 1.1-1

/var/log/messages on node 1:
Code:
Aug 29 06:25:01 hause03 rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="1859" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.
Aug 29 06:25:01 hause03 rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="1859" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.
/var/log/messages on node 2:
Code:
Aug 28 06:25:01 hause02 rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="1891" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.
Aug 28 08:16:18 hause02 kernel: device tap701i0 entered promiscuous mode
Aug 28 08:16:18 hause02 kernel: vmbr1: port 4(tap701i0) entering forwarding state
Aug 28 08:18:08 hause02 kernel: device tap702i0 entered promiscuous mode
Aug 28 08:18:08 hause02 kernel: vmbr1: port 5(tap702i0) entering forwarding state
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 08:41:27 hause02 corosync[2170]:   [QUORUM] Members[2]: 1 3
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 08:41:27 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 08:41:27 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 08:41:27 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.11) ; members(old:3 left:1)
Aug 28 08:41:27 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 09:22:35 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 09:22:35 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 09:22:35 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 09:22:35 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 09:22:35 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.11) ; members(old:2 left:0)
Aug 28 09:22:35 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 11:32:42 hause02 corosync[2170]:   [QUORUM] Members[2]: 1 3
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 11:32:42 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 11:32:42 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 11:32:42 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.11) ; members(old:3 left:1)
Aug 28 11:32:42 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 11:40:05 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 11:40:05 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 11:40:05 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 11:40:05 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 11:40:05 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.11) ; members(old:2 left:0)
Aug 28 11:40:05 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 12:35:38 hause02 corosync[2170]:   [QUORUM] Members[2]: 1 3
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 12:35:38 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 12:35:38 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 12:35:38 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.11) ; members(old:3 left:1)
Aug 28 12:35:38 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 12:40:58 hause02 kernel: svc: failed to register lockdv1 RPC service (errno 97).
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 12:55:16 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 12:55:16 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 12:55:16 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 12:55:16 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 12:55:16 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.11) ; members(old:2 left:0)
Aug 28 12:55:16 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 12:55:49 hause02 kernel: ct0 nfs: server 10.20.64.111 not responding, still trying
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 13:03:53 hause02 corosync[2170]:   [QUORUM] Members[2]: 2 3
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 13:03:53 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 13:03:53 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 13:03:53 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.13) ; members(old:3 left:1)
Aug 28 13:03:53 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 13:08:38 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 13:08:38 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 13:08:38 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 13:08:38 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 13:08:38 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.13) ; members(old:2 left:0)
Aug 28 13:08:38 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 13:49:33 hause02 corosync[2170]:   [QUORUM] Members[2]: 2 3
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 13:49:33 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 13:49:33 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 13:49:33 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.13) ; members(old:3 left:1)
Aug 28 13:49:33 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 13:52:09 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 13:52:09 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 13:52:09 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 13:52:09 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 13:52:09 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.13) ; members(old:2 left:0)
Aug 28 13:52:09 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 14:15:40 hause02 corosync[2170]:   [QUORUM] Members[2]: 2 3
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 14:15:40 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 14:15:40 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 14:15:40 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.13) ; members(old:3 left:1)
Aug 28 14:15:40 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 14:18:10 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 14:18:10 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 14:18:10 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 14:18:10 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 14:18:10 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.13) ; members(old:2 left:0)
Aug 28 14:18:10 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 14:29:58 hause02 corosync[2170]:   [QUORUM] Members[2]: 2 3
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 14:29:58 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 14:29:58 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 14:29:58 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.13) ; members(old:3 left:1)
Aug 28 14:29:58 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] CLM CONFIGURATION CHANGE
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] New Configuration:
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.12) 
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.13) 
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] Members Left:
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] Members Joined:
Aug 28 14:32:03 hause02 corosync[2170]:   [CLM   ] #011r(0) ip(10.20.64.11) 
Aug 28 14:32:03 hause02 corosync[2170]:   [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 28 14:32:03 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 14:32:03 hause02 corosync[2170]:   [QUORUM] Members[3]: 1 2 3
Aug 28 14:32:03 hause02 corosync[2170]:   [CPG   ] chosen downlist: sender r(0) ip(10.20.64.13) ; members(old:2 left:0)
Aug 28 14:32:03 hause02 corosync[2170]:   [MAIN  ] Completed service synchronization, ready to provide service.
/var/log/messages on node 3:
Code:
Aug 29 06:25:01 hause03 rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="1859" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.
Aug 29 06:25:01 hause03 rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="1859" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.
The time configuration is OK.

I removed the NFS datastore, but no luck.

What can I do?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!