PVE-zync issues?

K

killmasta93

Renowned Member
Aug 13, 2017
974
59
68
31
Hi
I was wondering if someone else has had this issue before, recently been getting this alert

Code: 
Connection closed by 192.168.3.252 port 22

which i find it odd because it was working for over a year, i thought it was the server i rebooted but i dont know what it could be the issue.

the server

Code: 
root@prometheus2:~# pveversion
pve-manager/6.2-4/9824574a (running kernel: 5.4.34-1-pve)

thank you
 
Hi,
can you connect normally via ssh? Is the network connection fine? Might also be worth to check the syslog on both nodes.
 
Thanks for the reply, so was checking the logs on the email i got

on the host that sends the error i only see this the part on the SSMTP is because it failed and sent me the error which is odd

Code: 
Mar 23 00:00:01 prometheus systemd[1]: Started Proxmox VE replication runner.
Mar 23 00:00:02 prometheus CRON[15378]: (root) CMD (pve-zsync sync --source 100 --dest xx.xx.xx.xxx:rpool/data/xxx --name bakolympus --maxsnap 27 --method ssh --source-user root --dest-user root)
Mar 23 00:00:02 prometheus CRON[15379]: (root) CMD (pve-zsync sync --source 101 --dest xx.xx.xx.xxx:rpool/data/xxx --name bakapolo --maxsnap 27 --method ssh --source-user root --dest-user root)
Mar 23 00:00:02 prometheus CRON[15381]: (root) CMD (pve-zsync sync --source 102 --dest xx.xx.xx.xxx:rpool/data/xxx --name bakatena --maxsnap 27 --method ssh --source-user root --dest-user root)
Mar 23 00:00:02 prometheus CRON[15380]: (root) CMD (pve-zsync sync --source 103 --dest xx.xx.xx.xxx:rpool/data/xxx --name bakhefesto --maxsnap 27 --method ssh --source-user root --dest-user root)
Mar 23 00:00:02 prometheus CRON[15382]: (root) CMD (if [ -d "/run/systemd/system" ]; then systemctl restart atop; else /usr/share/atop/atop.daily \& ; fi)
Mar 23 00:00:02 prometheus systemd[1]: Stopping Atop advanced performance monitor...
Mar 23 00:00:02 prometheus systemd[1]: Stopped Atop advanced performance monitor.
Mar 23 00:00:02 prometheus systemd[1]: Started Atop advanced performance monitor.
Mar 23 00:01:00 prometheus systemd[1]: Starting Proxmox VE replication runner...
Mar 23 00:01:01 prometheus systemd[1]: Started Proxmox VE replication runner.
Mar 23 00:02:00 prometheus systemd[1]: Starting Proxmox VE replication runner...
Mar 23 00:02:01 prometheus systemd[1]: Started Proxmox VE replication runner.
Mar 23 00:03:00 prometheus systemd[1]: Starting Proxmox VE replication runner...
Mar 23 00:03:01 prometheus systemd[1]: Started Proxmox VE replication runner.
Mar 23 00:03:35 prometheus sSMTP[14319]: Creating SSL connection to host
Mar 23 00:03:36 prometheus sSMTP[14319]: SSL connection using RSA_AES_256_CBC_SHA1
Mar 23 00:03:37 prometheus sSMTP[14319]: Sent mail for root@prometheus.xxx.local (221 2.0.0 Bye) uid=0 username=root outbytes=1141


on the server that receive the zfs data
Code: 
Mar 23 00:01:34 prometheus2 systemd[1]: session-1235.scope: Succeeded.
Mar 23 00:01:34 prometheus2 systemd[1]: Started Session 1236 of user root.
Mar 23 00:01:35 prometheus2 systemd[1]: session-1236.scope: Succeeded.
Mar 23 00:02:00 prometheus2 systemd[1]: Starting Proxmox VE replication runner...
Mar 23 00:02:01 prometheus2 systemd[1]: pvesr.service: Succeeded.
Mar 23 00:02:01 prometheus2 systemd[1]: Started Proxmox VE replication runner.
Mar 23 00:03:00 prometheus2 systemd[1]: Starting Proxmox VE replication runner...
Mar 23 00:03:01 prometheus2 systemd[1]: pvesr.service: Succeeded.
Mar 23 00:03:01 prometheus2 systemd[1]: Started Proxmox VE replication runner.
Mar 23 00:03:36 prometheus2 systemd[1]: Started Session 1237 of user root.
Mar 23 00:03:37 prometheus2 systemd[1]: session-1237.scope: Succeeded.
Mar 23 00:04:00 prometheus2 systemd[1]: Starting Proxmox VE replication runner...
Mar 23 00:04:01 prometheus2 systemd[1]: pvesr.service: Succeeded.
Mar 23 00:04:01 prometheus2 systemd[1]: Started Proxmox VE replication runner.
Mar 23 00:04:37 prometheus2 systemd[1]: session-1234.scope: Succeeded.
Mar 23 00:04:38 prometheus2 zed: eid=1306 class=history_event pool_guid=0x6A57A26488105950 
Mar 23 00:04:38 prometheus2 zed: eid=1307 class=history_event pool_guid=0x6A57A26488105950 
Mar 23 00:04:39 prometheus2 systemd[1]: Started Session 1238 of user root.
Mar 23 00:04:39 prometheus2 zed: eid=1308 class=history_event pool_guid=0x6A57A26488105950 
Mar 23 00:04:39 prometheus2 zed: eid=1309 class=history_event pool_guid=0x6A57A26488105950 
Mar 23 00:04:40 prometheus2 zed: eid=1310 class=history_event pool_guid=0x6A57A26488105950
 
What if you manually run a sync, i.e.
Code: 
pve-zsync sync --source 100 --dest xx.xx.xx.xxx:rpool/data/xxx --name bakolympus --maxsnap 27 --method ssh --source-user root --dest-user root

If the connection closed comes immediately, then the other log entries are unlikely to be related to pve-zsync as they happen a few minutes later.

Please check if your ssh configuration is correct and that you can connect between the servers.
 
thanks for the reply, so checking in more depth i ran another pve zync manually what it seems odd is that it get stucks sometimes
not always, after it get stuck i get the broken, i tried checking with wireshark but didnt find anything odd


not sure you mean ssh config?
10:07:36 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:37 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:38 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:39 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:40 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:41 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:42 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:43 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:44 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:45 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:46 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:47 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:48 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:49 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:50 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:51 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:52 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:53 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:54 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:55 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:56 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:57 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:58 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:07:59 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:00 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:01 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:02 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:03 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:04 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:05 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:06 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:07 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:08 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:09 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:10 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:11 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:12 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:13 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:14 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:15 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:16 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:17 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:18 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:19 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:20 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:21 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:22 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:23 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
10:08:24 56.3M rpool/data/vm-111-disk-1@rep_drpherculess_2021-04-01_09:59:36
Click to expand...
 
If it works sometimes, the ssh configuration is likely not the issue.

Maybe an issue with the network connection? What does zpool iostat -n 1 show when it gets stuck?

Could you also post the output of pveversion -v and zpool status on both source and target?
 
thanks for the reply, it seems to be an issue with the network im going to test again and post back to make sure its not a proxmox issue, currently
the host sending the zfs snaps have this version

Code: 
root@prometheus:~# pveversion -v
proxmox-ve: 5.4-1 (running kernel: 4.15.18-12-pve)
pve-manager: 5.4-3 (running version: 5.4-3/0a6eaa62)
pve-kernel-4.15: 5.3-3
pve-kernel-4.15.18-12-pve: 4.15.18-35
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-8
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-50
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-13
libpve-storage-perl: 5.0-41
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-25
pve-cluster: 5.0-36
pve-container: 2.0-37
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-19
pve-firmware: 2.0-6
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 2.12.1-3
pve-xtermjs: 3.12.0-1
pve-zsync: 2.0-3~bpo5
qemu-server: 5.0-50
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2

and the host receiving the snaps have this

Code: 
root@prometheus9:~# pveversion -v

proxmox-ve: 6.3-1 (running kernel: 5.4.73-1-pve)
pve-manager: 6.3-2 (running version: 6.3-2/22f57405)
pve-kernel-5.4: 6.3-1
pve-kernel-helper: 6.3-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.5
libproxmox-backup-qemu0: 1.0.2-1
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.2-6
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.3-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.5-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-3
pve-cluster: 6.2-1
pve-container: 3.3-1
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-7
pve-xtermjs: 4.7.0-3
qemu-server: 6.3-1
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.5-pve1
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom