PBS Sync job fails with SSL error after 8min up to 30mins

johen

New Member
Mar 27, 2022
7
0
1
HI

I have the problem that longer sync jobs fail after a random time and I'm currently out of ideas....
Different to this case https://forum.proxmox.com/threads/failed-backup.91772/
the ssl error appear after a random time between 8min - 30min. During this time backup snapshots gets successfully transfered and with starting the sync job again, I'm able to step by step sync the datastores...

thanks for any idea

best regards
JOchen

Both Backup Server are Version 2.2-3

2022-08-06T10:37:30+02:00: Starting datastore sync job 'pve0pbs:4TBext4:10TBext4::s-96faba2b-839d'
2022-08-06T10:37:30+02:00: sync datastore '10TBext4' from 'pve0pbs/4TBext4'
2022-08-06T10:37:30+02:00: ----
2022-08-06T10:37:30+02:00: Syncing datastore '4TBext4', root namespace into datastore '10TBext4', root namespace
2022-08-06T10:37:30+02:00: found 3 groups to sync
2022-08-06T10:37:30+02:00: re-sync snapshot ct/111/2022-08-06T00:32:09Z
2022-08-06T10:37:30+02:00: no data changes
2022-08-06T10:37:30+02:00: re-sync snapshot ct/111/2022-08-06T00:32:09Z done
2022-08-06T10:37:30+02:00: percentage done: 33.33% (1/3 groups)
2022-08-06T10:37:30+02:00: skipped: 16 snapshot(s) (2022-06-13T00:32:30Z .. 2022-08-05T00:32:04Z) older than the newest local snapshot
2022-08-06T10:37:30+02:00: re-sync snapshot vm/109/2022-08-06T00:30:00Z
2022-08-06T10:37:30+02:00: no data changes
2022-08-06T10:37:30+02:00: re-sync snapshot vm/109/2022-08-06T00:30:00Z done
2022-08-06T10:37:30+02:00: percentage done: 66.67% (2/3 groups)
2022-08-06T10:37:30+02:00: skipped: 16 snapshot(s) (2022-06-13T00:30:01Z .. 2022-08-05T00:30:02Z) older than the newest local snapshot
2022-08-06T10:37:30+02:00: re-sync snapshot vm/110/2022-07-24T00:30:07Z
2022-08-06T10:37:30+02:00: no data changes
2022-08-06T10:37:30+02:00: re-sync snapshot vm/110/2022-07-24T00:30:07Z done
2022-08-06T10:37:30+02:00: percentage done: 74.51% (2/3 groups, 4/17 snapshots in group #3)
2022-08-06T10:37:30+02:00: sync snapshot vm/110/2022-07-25T00:30:06Z
2022-08-06T10:37:30+02:00: sync archive qemu-server.conf.blob
2022-08-06T10:37:30+02:00: sync archive drive-scsi1.img.fidx
2022-08-06T10:39:42+02:00: downloaded 0 bytes (0.00 MiB/s)
2022-08-06T10:39:42+02:00: sync archive drive-scsi0.img.fidx
2022-08-06T10:42:05+02:00: downloaded 3992006310 bytes (26.78 MiB/s)
2022-08-06T10:42:05+02:00: sync archive drive-efidisk0.img.fidx
2022-08-06T10:42:05+02:00: downloaded 0 bytes (0.00 MiB/s)
2022-08-06T10:42:05+02:00: got backup log file "client.log.blob"
2022-08-06T10:42:05+02:00: sync snapshot vm/110/2022-07-25T00:30:06Z done
2022-08-06T10:42:05+02:00: percentage done: 76.47% (2/3 groups, 5/17 snapshots in group #3)
2022-08-06T10:42:05+02:00: sync snapshot vm/110/2022-07-26T00:30:07Z
2022-08-06T10:42:05+02:00: sync archive qemu-server.conf.blob
2022-08-06T10:42:05+02:00: sync archive drive-scsi1.img.fidx
2022-08-06T10:42:09+02:00: downloaded 100695474 bytes (26.31 MiB/s)
2022-08-06T10:42:09+02:00: sync archive drive-scsi0.img.fidx
2022-08-06T10:44:33+02:00: downloaded 4773108008 bytes (31.78 MiB/s)
2022-08-06T10:44:33+02:00: sync archive drive-efidisk0.img.fidx
2022-08-06T10:44:33+02:00: downloaded 0 bytes (0.00 MiB/s)
2022-08-06T10:44:33+02:00: got backup log file "client.log.blob"
2022-08-06T10:44:33+02:00: sync snapshot vm/110/2022-07-26T00:30:07Z done
2022-08-06T10:44:33+02:00: percentage done: 78.43% (2/3 groups, 6/17 snapshots in group #3)
2022-08-06T10:44:33+02:00: sync snapshot vm/110/2022-07-27T00:30:08Z
2022-08-06T10:44:33+02:00: sync archive qemu-server.conf.blob
2022-08-06T10:44:33+02:00: sync archive drive-scsi1.img.fidx
2022-08-06T10:44:36+02:00: downloaded 63646586 bytes (26.23 MiB/s)
2022-08-06T10:44:36+02:00: sync archive drive-scsi0.img.fidx
2022-08-06T10:45:34+02:00: percentage done: 80.39% (2/3 groups, 7/17 snapshots in group #3)
2022-08-06T10:45:34+02:00: sync group vm/110 failed - error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:676:
2022-08-06T10:45:34+02:00: Finished syncing namespace , current progress: 2 groups, 7 snapshots
2022-08-06T10:45:34+02:00: TASK ERROR: sync failed with some errors.
 
Last edited:

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
8,551
1,101
164
34
Vienna
is there some firewall in between that could be the issue?
 

johen

New Member
Mar 27, 2022
7
0
1
is there some firewall in between that could be the issue?
Hi Dominik,
thanks for your reply

no I don't have any firewall between the PBS.
to me the error behaviour doesn't fit to a firewall blocking something, as the error sometimes doesn't appear for 30min and is random

best regards
Jochen
 

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
8,551
1,101
164
34
Vienna
did you already check the network stack (cables/switches) and the hardware? (memory test, etc.)
the error can have many causes, e.g. the issue here: https://github.com/openssl/openssl/issues/11727
mentions that in that case it was a 'bad' router with port forwarding (thats why i asked if there is a firewall)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!