I installed Proxmox 4.0 beta1 on three nodes and updated to latest version.
Creating a cluster on the master node went without problems.
Adding other two nodes returns an error:
root pve2:~# pvecm add 10.25.3.45
root 10.25.3.45's password:
unable to copy ssh ID
pvecm status on master node:
root pve1:/var/log# pvecm status
Quorum information
------------------
Date: Fri Aug 21 22:56:45 2015
Quorum provider: corosync_votequorum
Nodes: 1
Node ID: 0x00000001
Ring ID: 4
Quorate: Yes
Votequorum information
----------------------
Expected votes: 1
Highest expected: 1
Total votes: 1
Quorum: 1
Flags: Quorate
Membership information
----------------------
Nodeid Votes Name
0x00000001 1 10.25.3.45 (local)
Also syslog contains these errors:
root pve1:/var/log# tail syslog
Aug 21 22:41:46 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:46 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:46 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:46 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
root pve1:/var/log# pveversion
pve-manager/4.0-26/5d4a615b (running kernel: 4.1.3-1-pve)
root pve1:/var/log# service --status-all
[ + ] apparmor
[ + ] atd
[ - ] bootlogd
[ - ] bootlogs
[ - ] bootmisc.sh
[ - ] checkfs.sh
[ - ] checkroot-bootclean.sh
[ - ] checkroot.sh
[ + ] console-setup
[ + ] cron
[ + ] dbus
[ + ] hdparm
[ - ] hostname.sh
[ - ] hwclock.sh
[ + ] kbd
[ + ] keyboard-setup
[ - ] killprocs
[ + ] kmod
[ - ] lvm2
[ - ] motd
[ - ] mountall-bootclean.sh
[ - ] mountall.sh
[ - ] mountdevsubfs.sh
[ - ] mountkernfs.sh
[ - ] mountnfs-bootclean.sh
[ - ] mountnfs.sh
[ + ] networking
[ + ] nfs-common
[ + ] open-iscsi
[ + ] postfix
[ + ] procps
[ + ] pve-cluster
[ + ] pve-firewall
[ + ] pve-manager
[ + ] pvedaemon
[ + ] pvefw-logger
[ + ] pveproxy
[ + ] pvestatd
[ + ] rc.local
[ - ] rmnologin
[ + ] rpcbind
[ + ] rrdcached
[ - ] rsync
[ + ] rsyslog
[ - ] sendsigs
[ + ] spiceproxy
[ + ] ssh
[ - ] stop-bootlogd
[ - ] stop-bootlogd-single
[ + ] udev
[ + ] udev-finish
[ - ] umountfs
[ - ] umountiscsi.sh
[ - ] umountnfs.sh
[ - ] umountroot
[ + ] urandom
[ - ] x11-common
[ + ] zfs-import
[ + ] zfs-mount
[ + ] zfs-share
[ + ] zfs-zed
I am not sure where else to look for clues.
Creating a cluster on the master node went without problems.
Adding other two nodes returns an error:
root pve2:~# pvecm add 10.25.3.45
root 10.25.3.45's password:
unable to copy ssh ID
pvecm status on master node:
root pve1:/var/log# pvecm status
Quorum information
------------------
Date: Fri Aug 21 22:56:45 2015
Quorum provider: corosync_votequorum
Nodes: 1
Node ID: 0x00000001
Ring ID: 4
Quorate: Yes
Votequorum information
----------------------
Expected votes: 1
Highest expected: 1
Total votes: 1
Quorum: 1
Flags: Quorate
Membership information
----------------------
Nodeid Votes Name
0x00000001 1 10.25.3.45 (local)
Also syslog contains these errors:
root pve1:/var/log# tail syslog
Aug 21 22:41:46 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:46 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:46 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:46 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
Aug 21 22:41:56 pve1 pmxcfs[1369]: [status] crit: cpg_send_message failed: 9
root pve1:/var/log# pveversion
pve-manager/4.0-26/5d4a615b (running kernel: 4.1.3-1-pve)
root pve1:/var/log# service --status-all
[ + ] apparmor
[ + ] atd
[ - ] bootlogd
[ - ] bootlogs
[ - ] bootmisc.sh
[ - ] checkfs.sh
[ - ] checkroot-bootclean.sh
[ - ] checkroot.sh
[ + ] console-setup
[ + ] cron
[ + ] dbus
[ + ] hdparm
[ - ] hostname.sh
[ - ] hwclock.sh
[ + ] kbd
[ + ] keyboard-setup
[ - ] killprocs
[ + ] kmod
[ - ] lvm2
[ - ] motd
[ - ] mountall-bootclean.sh
[ - ] mountall.sh
[ - ] mountdevsubfs.sh
[ - ] mountkernfs.sh
[ - ] mountnfs-bootclean.sh
[ - ] mountnfs.sh
[ + ] networking
[ + ] nfs-common
[ + ] open-iscsi
[ + ] postfix
[ + ] procps
[ + ] pve-cluster
[ + ] pve-firewall
[ + ] pve-manager
[ + ] pvedaemon
[ + ] pvefw-logger
[ + ] pveproxy
[ + ] pvestatd
[ + ] rc.local
[ - ] rmnologin
[ + ] rpcbind
[ + ] rrdcached
[ - ] rsync
[ + ] rsyslog
[ - ] sendsigs
[ + ] spiceproxy
[ + ] ssh
[ - ] stop-bootlogd
[ - ] stop-bootlogd-single
[ + ] udev
[ + ] udev-finish
[ - ] umountfs
[ - ] umountiscsi.sh
[ - ] umountnfs.sh
[ - ] umountroot
[ + ] urandom
[ - ] x11-common
[ + ] zfs-import
[ + ] zfs-mount
[ + ] zfs-share
[ + ] zfs-zed
I am not sure where else to look for clues.