Please provide the output of pveversion -v
.
What are the exact errors you're getting?
Please also provide your /etc/hosts file for all 3 nodes.
Code:
~# pveversion -v
proxmox-ve: 7.1-2 (running kernel: 5.13.19-6-pve)
pve-manager: 7.1-12 (running version: 7.1-12/b3c09de3)
pve-kernel-helper: 7.2-2
pve-kernel-5.13: 7.1-9
pve-kernel-5.11: 7.0-10
pve-kernel-5.4: 6.4-5
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-5-pve: 5.13.19-13
pve-kernel-5.13.19-4-pve: 5.13.19-9
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.13.19-1-pve: 5.13.19-3
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-5-pve: 5.11.22-10
pve-kernel-5.11.22-4-pve: 5.11.22-9
pve-kernel-5.11.22-3-pve: 5.11.22-7
pve-kernel-5.11.22-2-pve: 5.11.22-4
pve-kernel-5.4.128-1-pve: 5.4.128-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-7
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.1-5
libpve-guest-common-perl: 4.1-2
libpve-http-server-perl: 4.1-1
libpve-storage-perl: 7.1-2
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.12-1
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.1.6-1
proxmox-backup-file-restore: 2.1.6-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-10
pve-cluster: 7.1-3
pve-container: 4.1-5
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.4-1
pve-ha-manager: 3.3-4
pve-i18n: 2.6-3
pve-qemu-kvm: 6.2.0-5
pve-xtermjs: 4.16.0-1
qemu-server: 7.1-5
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.4-pve1
Please provide the output of pveversion -v
.
What are the exact errors you're getting?
Please also provide your /etc/hosts file for all 3 nodes.
Code:
Host 1
127.0.0.1 localhost.localdomain localhost
10.0.X.X bob bob
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
Host 2
127.0.0.1 localhost.localdomain localhost
10.0.x.x bob-midi bob-midi
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
Host 3
127.0.0.1 localhost.localdomain localhost
10.0.x.x bob-mini bob-mini
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
errors:
2022-05-03 18:16:01 102-0: start replication job
2022-05-03 18:16:01 102-0: guest => VM 102, running => 13474
2022-05-03 18:16:01 102-0: volumes => bob-Local-VMS:vm-102-disk-0
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) Someone could be eavesdropping on you right now (man-in-the-middle attack)!
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) It is also possible that a host key has just been changed.
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) The fingerprint for the RSA key sent by the remote host is
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) *****.
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) Please contact your system administrator.
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) Add correct host key in /root/.ssh/known_hosts to get rid of this message.
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) Offending RSA key in /etc/ssh/ssh_known_hosts:10
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) remove with:
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R "bob-midi"
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) RSA host key for bob-midi has changed and you have requested strict checking.
2022-05-03 18:16:01 102-0: (remote_prepare_local_job) Host key verification failed.
When running
ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R "bob-midi"
next error is:
2022-05-03 18:47:01 102-0: start replication job
2022-05-03 18:47:01 102-0: guest => VM 102, running => 13474
2022-05-03 18:47:01 102-0: volumes => bob-Local-VMS:vm-102-disk-0
2022-05-03 18:47:01 102-0: (remote_prepare_local_job) Host key verification faile
what is weird - host 2 to 3 does not work but 3 to 2 works if you do the refreshing of the keys and also do pvecm updatecerts