Hello,I
I have a 4 node cluster with the following version:
root@proxmox-cluster-01:~# pveversion -v
proxmox-ve-2.6.32: 3.1-114 (running kernel: 2.6.32-26-pve)
pve-manager: 3.1-21 (running version: 3.1-21/93bf03d4)
pve-kernel-2.6.32-26-pve: 2.6.32-114
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.5-1
pve-cluster: 3.0-12
qemu-server: 3.1-15
pve-firmware: 1.1-2
libpve-common-perl: 3.0-14
libpve-access-control: 3.0-11
libpve-storage-perl: 3.0-19
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-6
vzctl: 4.0-1pve5
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.7-6
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.2-1
I've installed the new server with the older packages and it matches the old/existing servers:
root@proxmox-cluster-05:~# pveversion -v
proxmox-ve-2.6.32: 3.3-147 (running kernel: 2.6.32-26-pve)
pve-manager: 3.1-21 (running version: 3.1-21/93bf03d4)
pve-kernel-2.6.32-37-pve: 2.6.32-147
pve-kernel-2.6.32-34-pve: 2.6.32-140
pve-kernel-2.6.32-26-pve: 2.6.32-114
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.5-1
pve-cluster: 3.0-12
qemu-server: 3.1-15
pve-firmware: 1.1-2
libpve-common-perl: 3.0-14
libpve-access-control: 3.0-11
libpve-storage-perl: 3.0-19
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-6
vzctl: 4.0-1pve5
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.7-6
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.2-1
When I try to join the cluster it will try to start CMAN and it fails with:
copy corosync auth key
stopping pve-cluster service
Stopping pve cluster filesystem: pve-cluster.
backup old database
Starting pve cluster filesystem : pve-cluster.
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... corosync died with signal: 11 Check cluster logs for details
[FAILED]
waiting for quorum...
In the syslog:
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [MAIN ] Corosync Cluster Engine ('1.4.5'): started and ready to provide service.
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [MAIN ] Corosync built-in features: nss
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [MAIN ] Successfully read config from /etc/cluster/cluster.conf
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [MAIN ] Successfully parsed cman config
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [MAIN ] Successfully configured openais services to load
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [TOTEM ] Initializing transport (UDP/IP Multicast).
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [TOTEM ] The network interface is down.
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:41 proxmox-cluster-05 pmxcfs[17567]: [quorum] crit: quorum_initialize failed: 6
Feb 17 12:10:41 proxmox-cluster-05 pmxcfs[17567]: [confdb] crit: confdb_initialize failed: 6
Feb 17 12:10:41 proxmox-cluster-05 pmxcfs[17567]: [dcdb] crit: cpg_initialize failed: 6
Feb 17 12:10:41 proxmox-cluster-05 pmxcfs[17567]: [dcdb] crit: cpg_initialize failed: 6
I've also tried to stop services, remove all cluster folders and rety but nogo. Also tried with the latest version and latest kernel.
I have a 4 node cluster with the following version:
root@proxmox-cluster-01:~# pveversion -v
proxmox-ve-2.6.32: 3.1-114 (running kernel: 2.6.32-26-pve)
pve-manager: 3.1-21 (running version: 3.1-21/93bf03d4)
pve-kernel-2.6.32-26-pve: 2.6.32-114
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.5-1
pve-cluster: 3.0-12
qemu-server: 3.1-15
pve-firmware: 1.1-2
libpve-common-perl: 3.0-14
libpve-access-control: 3.0-11
libpve-storage-perl: 3.0-19
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-6
vzctl: 4.0-1pve5
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.7-6
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.2-1
I've installed the new server with the older packages and it matches the old/existing servers:
root@proxmox-cluster-05:~# pveversion -v
proxmox-ve-2.6.32: 3.3-147 (running kernel: 2.6.32-26-pve)
pve-manager: 3.1-21 (running version: 3.1-21/93bf03d4)
pve-kernel-2.6.32-37-pve: 2.6.32-147
pve-kernel-2.6.32-34-pve: 2.6.32-140
pve-kernel-2.6.32-26-pve: 2.6.32-114
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.5-1
pve-cluster: 3.0-12
qemu-server: 3.1-15
pve-firmware: 1.1-2
libpve-common-perl: 3.0-14
libpve-access-control: 3.0-11
libpve-storage-perl: 3.0-19
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-6
vzctl: 4.0-1pve5
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.7-6
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.2-1
When I try to join the cluster it will try to start CMAN and it fails with:
copy corosync auth key
stopping pve-cluster service
Stopping pve cluster filesystem: pve-cluster.
backup old database
Starting pve cluster filesystem : pve-cluster.
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... corosync died with signal: 11 Check cluster logs for details
[FAILED]
waiting for quorum...
In the syslog:
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [MAIN ] Corosync Cluster Engine ('1.4.5'): started and ready to provide service.
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [MAIN ] Corosync built-in features: nss
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [MAIN ] Successfully read config from /etc/cluster/cluster.conf
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [MAIN ] Successfully parsed cman config
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [MAIN ] Successfully configured openais services to load
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [TOTEM ] Initializing transport (UDP/IP Multicast).
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Feb 17 12:10:37 proxmox-cluster-05 corosync[18082]: [TOTEM ] The network interface is down.
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:38 proxmox-cluster-05 pmxcfs[17567]: [status] crit: cpg_send_message failed: 9
Feb 17 12:10:41 proxmox-cluster-05 pmxcfs[17567]: [quorum] crit: quorum_initialize failed: 6
Feb 17 12:10:41 proxmox-cluster-05 pmxcfs[17567]: [confdb] crit: confdb_initialize failed: 6
Feb 17 12:10:41 proxmox-cluster-05 pmxcfs[17567]: [dcdb] crit: cpg_initialize failed: 6
Feb 17 12:10:41 proxmox-cluster-05 pmxcfs[17567]: [dcdb] crit: cpg_initialize failed: 6
I've also tried to stop services, remove all cluster folders and rety but nogo. Also tried with the latest version and latest kernel.