Only one Backup of remote location fails

Z

Zaphood

Member
Proxmox Subscriber
Nov 17, 2021
9
0
6
57
Hi,

I am running proxmox-backup-server: 3.2.6-1 which is used to back up 2 Proxmox Hosts with LXC containers as well as VMs. One Server is local, the other one is remote, connected via a VPN.

On the local Server all is working well. On the remote server (which hosts one VM and one LXC Container) on the other hand, only the VM is backed up without problems. Everytime the backup for the LXC Container is startet, it fails with the following errors:

Code: 
INFO:
INFO: starting new backup job: vzdump 315 --storage Backup-Proxmox2 --mode snapshot --mailnotification always --notification-mode legacy-sendmail --fleecing 0 --bwlimit 20480 --mailto frank.beckereit@gmx.net --node proxmox3 --notes-template '{{guestname}}' --prune-backups 'keep-daily=3,keep-hourly=1,keep-last=1,keep-monthly=1' --quiet 1
INFO: Starting Backup of VM 315 (lxc)
INFO: Backup started at 2024-06-30 01:00:00
INFO: status = running
INFO: CT Name: LMS2
INFO: including mount point rootfs ('/') in backup
INFO: backup mode: snapshot
INFO: bandwidth limit: 20480 KB/s
INFO: ionice priority: 7
INFO: create storage snapshot 'vzdump'
  Logical volume "snap_vm-315-disk-0_vzdump" created.
INFO: creating Proxmox Backup Server archive 'ct/315/2024-06-29T23:00:00Z'
INFO: set max number of entries in memory for file-based backups to 1048576
INFO: run: lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup --crypt-mode=none pct.conf:/var/tmp/vzdumptmp4012760_315/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --skip-lost-and-found --exclude=/tmp/?* --exclude=/var/tmp/?* --exclude=/var/run/?*.pid --backup-type ct --backup-id 315 --backup-time 1719702000 --entries-max 1048576 --repository root@pam@192.168.1.21:Backup1 --ns PROXMOX3
INFO: Starting backup: [PROXMOX3]:ct/315/2024-06-29T23:00:00Z
INFO: Client name: proxmox3
INFO: Starting backup protocol: Sun Jun 30 01:00:01 2024
INFO: No previous manifest available.
INFO: Upload config file '/var/tmp/vzdumptmp4012760_315/etc/vzdump/pct.conf' to 'root@pam@192.168.1.21:8007:Backup1' as pct.conf.blob
INFO: Upload directory '/mnt/vzsnap0' to 'root@pam@192.168.1.21:8007:Backup1' as root.pxar.didx
INFO: HTTP/2.0 connection failed
INFO: catalog upload error - channel closed
INFO: Error: error:0A0003FC:SSL routines:ssl3_read_bytes:sslv3 alert bad record mac:../ssl/record/rec_layer_s3.c:1586:SSL alert number 20
INFO: cleanup temporary 'vzdump' snapshot
  Logical volume "snap_vm-315-disk-0_vzdump" successfully removed.
ERROR: Backup of VM 315 failed - command 'lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup '--crypt-mode=none' pct.conf:/var/tmp/vzdumptmp4012760_315/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --skip-lost-and-found '--exclude=/tmp/?*' '--exclude=/var/tmp/?*' '--exclude=/var/run/?*.pid' --backup-type ct --backup-id 315 --backup-time 1719702000 --entries-max 1048576 --repository root@pam@192.168.1.21:Backup1 --ns PROXMOX3' failed: exit code 255
INFO: Failed at 2024-06-30 01:38:25
INFO: Backup job finished with errors
INFO: notified via target `<xxxxxxxx>`
TASK ERROR: job errors


Not sure what that "bad record" means or why the backup ultimately fails.

Thanks for any hint
Frank
 
Last edited:
Zaphood said:
Hi,

I am running proxmox-backup-server: 3.2.6-1 which is used to back up 2 Proxmox Hosts with LXC containers as well as VMs. One Server is local, the other one is remote, connected via a VPN.

On the local Server all is working well. On the remote server (which hosts one VM and one LXC Container) on the other hand, only the VM is backed up without problems. Everytime the backup for the LXC Container is startet, it fails with the following errors:

Code: 
INFO:
INFO: starting new backup job: vzdump 315 --storage Backup-Proxmox2 --mode snapshot --mailnotification always --notification-mode legacy-sendmail --fleecing 0 --bwlimit 20480 --mailto frank.beckereit@gmx.net --node proxmox3 --notes-template '{{guestname}}' --prune-backups 'keep-daily=3,keep-hourly=1,keep-last=1,keep-monthly=1' --quiet 1
INFO: Starting Backup of VM 315 (lxc)
INFO: Backup started at 2024-06-30 01:00:00
INFO: status = running
INFO: CT Name: LMS2
INFO: including mount point rootfs ('/') in backup
INFO: backup mode: snapshot
INFO: bandwidth limit: 20480 KB/s
INFO: ionice priority: 7
INFO: create storage snapshot 'vzdump'
  Logical volume "snap_vm-315-disk-0_vzdump" created.
INFO: creating Proxmox Backup Server archive 'ct/315/2024-06-29T23:00:00Z'
INFO: set max number of entries in memory for file-based backups to 1048576
INFO: run: lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup --crypt-mode=none pct.conf:/var/tmp/vzdumptmp4012760_315/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --skip-lost-and-found --exclude=/tmp/?* --exclude=/var/tmp/?* --exclude=/var/run/?*.pid --backup-type ct --backup-id 315 --backup-time 1719702000 --entries-max 1048576 --repository root@pam@192.168.1.21:Backup1 --ns PROXMOX3
INFO: Starting backup: [PROXMOX3]:ct/315/2024-06-29T23:00:00Z
INFO: Client name: proxmox3
INFO: Starting backup protocol: Sun Jun 30 01:00:01 2024
INFO: No previous manifest available.
INFO: Upload config file '/var/tmp/vzdumptmp4012760_315/etc/vzdump/pct.conf' to 'root@pam@192.168.1.21:8007:Backup1' as pct.conf.blob
INFO: Upload directory '/mnt/vzsnap0' to 'root@pam@192.168.1.21:8007:Backup1' as root.pxar.didx
INFO: HTTP/2.0 connection failed
INFO: catalog upload error - channel closed
INFO: Error: error:0A0003FC:SSL routines:ssl3_read_bytes:sslv3 alert bad record mac:../ssl/record/rec_layer_s3.c:1586:SSL alert number 20
INFO: cleanup temporary 'vzdump' snapshot
  Logical volume "snap_vm-315-disk-0_vzdump" successfully removed.
ERROR: Backup of VM 315 failed - command 'lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup '--crypt-mode=none' pct.conf:/var/tmp/vzdumptmp4012760_315/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --skip-lost-and-found '--exclude=/tmp/?*' '--exclude=/var/tmp/?*' '--exclude=/var/run/?*.pid' --backup-type ct --backup-id 315 --backup-time 1719702000 --entries-max 1048576 --repository root@pam@192.168.1.21:Backup1 --ns PROXMOX3' failed: exit code 255
INFO: Failed at 2024-06-30 01:38:25
INFO: Backup job finished with errors
INFO: notified via target `<xxxxxxxx>`
TASK ERROR: job errors


Not sure what that "bad record" means or why the backup ultimately fails.

Thanks for any hint
Frank
Click to expand...
Hi,
seems to be an issue with the certificate, as the TLS handshake seems to fail when upgrading to the HTTP/2 connection. Any errors in the systemd journal of the Proxmox Backup Server?
 
Chris said:
Hi,
seems to be an issue with the certificate, as the TLS handshake seems to fail when upgrading to the HTTP/2 connection. Any errors in the systemd journal of the Proxmox Backup Server?
Click to expand...
Thanks for the hint.

Looking at the log of the backup-server, it seems to start fine at 1am and is successful in uploading many chunks. After app. 38min, it then fails. Attached just the last lines of that task (hope that tells you anyting?)

Code: 
Jun 30 01:37:29 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4831163 bytes, 67d586bbcbc25dd0420d5b2d360c3aa781a8ecc562b24ef39cde45a431ae51ca
Jun 30 01:37:34 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 5501835 bytes, b55b5c8354d2f8ceef40b4a0c352f42a44d12ba568b39b24913274ebfba03343
Jun 30 01:37:36 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1777399 bytes, e34719af9d7938d83eb6b407f4ea92bc5cee4cda98cbee399ab0d9fee8962c88
Jun 30 01:37:37 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 10319205 bytes, da9f58d5058a721255b0dd9069e868757af382bf08b8f047405865dfa61d1970
Jun 30 01:37:39 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:39 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:39 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 5493681 bytes, 8f6ee856325b2d1f5439313037e7c703cc5dcc9f18e1d7647d743b181055b1dc
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4887442 bytes, 1f5da8ca58aac575cd249939b1776865d24602da27a343347e0672ac07fbf7fa
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4995457 bytes, 4667334874587625afafaee1417919ac71c0c7ff346d188ca7526fb33ec2c3b6
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4376033 bytes, ed385c506f9eb83227d17d287cdfbf0ac630c86032a10ebfee703d9a348a38c7
Jun 30 01:37:44 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 2057662 bytes, 1d573d9cce1fa6d4ab8842ac32db84ddf8bcfcbe593165765d3654a39bb77a4e
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 2853955 bytes, 0b1f6b57ddab16dee73b99e178059b97a92443e8bb949d4af71ae4abe43ec478
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4225551 bytes, cb9b13e541cd863771f2eb93a4bd9abc9cda81f0758c8404bf0a95e9dc9a16a5
Jun 30 01:37:50 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:50 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:51 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 10642678 bytes, d40c2e0687686869fdf950d0467c84cc678f0459d0de98f2dbb358936e9ed351
Jun 30 01:37:52 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1185603 bytes, 17578d110d04c04d338186b5eda5e1975fc35ef8bc9a9e43e58831d9a9d38df5
Jun 30 01:37:53 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 2889801 bytes, 5367741644ed8f461dd674e14f53022eb895b6ce7f4dddd16d3a30c662942a53
Jun 30 01:37:55 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:55 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:55 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1568484 bytes, 6493764e5028a023024d4984011193a49189e9dffcf00890e2883cf4b1ab2fec
Jun 30 01:37:55 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 8627285 bytes, a32eb43c07ff1b4863a93f423c09ced279d0a745e22c8db4b557b74cffda2b6b
Jun 30 01:37:57 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1955436 bytes, 6357b23a438d8e905f51b1a900e7e0a178afd836329104345e63b3262fa24017
Jun 30 01:37:59 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 7326596 bytes, 82df974cd22065524f706bbd6bd8274cbe80fbd921457d1fb39c7eea3906fb29
Jun 30 01:38:01 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:01 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:01 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 2472030 bytes, 5369637234c651a8b31a1bfeb16e08be8862d4a0fbf2431187f2d038536ed3fc
Jun 30 01:38:03 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:03 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:03 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:04 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 7600209 bytes, e9727000eb26cf0b386bb5ba8f4a7ade3df397bddd6eb2f2b55d6ebd3fb90974
Jun 30 01:38:06 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1195812 bytes, 88dcbc470e6bde35096389dd6a2ff7e25afdb8d48a8e9ad82a1a1c6c64dd88c5
Jun 30 01:38:08 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 11875725 bytes, 796819de365391e54e8ff982424ac166f713f7112724c883008752ef184ceeb0
Jun 30 01:38:09 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4193559 bytes, 4aedb2439d45d00939b8c167b3acb1a268512b42cda13bc95c373cc8f30fdce0
Jun 30 01:38:13 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:13 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:13 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:13 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 3841130 bytes, 55e06efe47326a7856e7d271a9c12c496f9e58a9c18de898daa9c0bee7ced6e0
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 7094111 bytes, 643d50895b8db422d87a82196168b2165b0487e9b4013933f08c62b68b73f692
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4721166 bytes, 5db2cecc9bc7103bc15252a89fc3385c451759a0c9a9ef2f0ca00da1359eef51
Jun 30 01:38:15 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1712155 bytes, e19c9e1abb21dfb56e0727a2399f134fa2d237ed692d2aad481b607cc58627d9
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1590289 bytes, 585d3307e43fb90c2dd935e3f7e6a5964877a369a48e6b12a3d41473ab75e744
Jun 30 01:38:18 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 2923166 bytes, 343e7c96777f174573009a505f9e158ce2078c5fea804ec18de08975ae7f4a93
Jun 30 01:38:20 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1572763 bytes, b26e9fcc32edd26747948e452d83e74739cae0b4709c2ff92bbac7e55023b621
Jun 30 01:38:21 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1879596 bytes, dd5dbb62a20c6d704491277022fc6f858f087fe38d0d9fdef6aa81e8be176927
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: backup failed: connection error: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: removing failed backup
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: TASK ERROR: connection error: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: removing backup snapshot "/mnt/datastore/Backup1/ns/PROXMOX3/ct/315/2024-06-29T23:00:00Z"
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:47:12 proxmox2 proxmox-backup-proxy[14513]: write rrd data back to disk
Jun 30 01:47:12 proxmox2 proxmox-backup-proxy[14513]: starting rrd data sync
Jun 30 01:47:12 proxmox2 proxmox-backup-proxy[14513]: rrd journal successfully committed (25 files in 0.008 seconds)
 
Zaphood said:
Thanks for the hint.

Looking at the log of the backup-server, it seems to start fine at 1am and is successful in uploading many chunks. After app. 38min, it then fails. Attached just the last lines of that task (hope that tells you anyting?)

Code: 
Jun 30 01:37:29 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4831163 bytes, 67d586bbcbc25dd0420d5b2d360c3aa781a8ecc562b24ef39cde45a431ae51ca
Jun 30 01:37:34 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 5501835 bytes, b55b5c8354d2f8ceef40b4a0c352f42a44d12ba568b39b24913274ebfba03343
Jun 30 01:37:36 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1777399 bytes, e34719af9d7938d83eb6b407f4ea92bc5cee4cda98cbee399ab0d9fee8962c88
Jun 30 01:37:37 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 10319205 bytes, da9f58d5058a721255b0dd9069e868757af382bf08b8f047405865dfa61d1970
Jun 30 01:37:39 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:39 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:39 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 5493681 bytes, 8f6ee856325b2d1f5439313037e7c703cc5dcc9f18e1d7647d743b181055b1dc
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4887442 bytes, 1f5da8ca58aac575cd249939b1776865d24602da27a343347e0672ac07fbf7fa
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4995457 bytes, 4667334874587625afafaee1417919ac71c0c7ff346d188ca7526fb33ec2c3b6
Jun 30 01:37:40 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4376033 bytes, ed385c506f9eb83227d17d287cdfbf0ac630c86032a10ebfee703d9a348a38c7
Jun 30 01:37:44 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 2057662 bytes, 1d573d9cce1fa6d4ab8842ac32db84ddf8bcfcbe593165765d3654a39bb77a4e
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 2853955 bytes, 0b1f6b57ddab16dee73b99e178059b97a92443e8bb949d4af71ae4abe43ec478
Jun 30 01:37:49 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4225551 bytes, cb9b13e541cd863771f2eb93a4bd9abc9cda81f0758c8404bf0a95e9dc9a16a5
Jun 30 01:37:50 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:50 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:51 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 10642678 bytes, d40c2e0687686869fdf950d0467c84cc678f0459d0de98f2dbb358936e9ed351
Jun 30 01:37:52 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1185603 bytes, 17578d110d04c04d338186b5eda5e1975fc35ef8bc9a9e43e58831d9a9d38df5
Jun 30 01:37:53 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 2889801 bytes, 5367741644ed8f461dd674e14f53022eb895b6ce7f4dddd16d3a30c662942a53
Jun 30 01:37:55 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:55 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:37:55 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1568484 bytes, 6493764e5028a023024d4984011193a49189e9dffcf00890e2883cf4b1ab2fec
Jun 30 01:37:55 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 8627285 bytes, a32eb43c07ff1b4863a93f423c09ced279d0a745e22c8db4b557b74cffda2b6b
Jun 30 01:37:57 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1955436 bytes, 6357b23a438d8e905f51b1a900e7e0a178afd836329104345e63b3262fa24017
Jun 30 01:37:59 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 7326596 bytes, 82df974cd22065524f706bbd6bd8274cbe80fbd921457d1fb39c7eea3906fb29
Jun 30 01:38:01 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:01 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:01 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 2472030 bytes, 5369637234c651a8b31a1bfeb16e08be8862d4a0fbf2431187f2d038536ed3fc
Jun 30 01:38:03 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:03 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:03 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:04 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 7600209 bytes, e9727000eb26cf0b386bb5ba8f4a7ade3df397bddd6eb2f2b55d6ebd3fb90974
Jun 30 01:38:06 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1195812 bytes, 88dcbc470e6bde35096389dd6a2ff7e25afdb8d48a8e9ad82a1a1c6c64dd88c5
Jun 30 01:38:08 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 11875725 bytes, 796819de365391e54e8ff982424ac166f713f7112724c883008752ef184ceeb0
Jun 30 01:38:09 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4193559 bytes, 4aedb2439d45d00939b8c167b3acb1a268512b42cda13bc95c373cc8f30fdce0
Jun 30 01:38:13 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:13 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:13 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:13 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 3841130 bytes, 55e06efe47326a7856e7d271a9c12c496f9e58a9c18de898daa9c0bee7ced6e0
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 7094111 bytes, 643d50895b8db422d87a82196168b2165b0487e9b4013933f08c62b68b73f692
Jun 30 01:38:14 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 4721166 bytes, 5db2cecc9bc7103bc15252a89fc3385c451759a0c9a9ef2f0ca00da1359eef51
Jun 30 01:38:15 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1712155 bytes, e19c9e1abb21dfb56e0727a2399f134fa2d237ed692d2aad481b607cc58627d9
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk
Jun 30 01:38:17 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1590289 bytes, 585d3307e43fb90c2dd935e3f7e6a5964877a369a48e6b12a3d41473ab75e744
Jun 30 01:38:18 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 2923166 bytes, 343e7c96777f174573009a505f9e158ce2078c5fea804ec18de08975ae7f4a93
Jun 30 01:38:20 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1572763 bytes, b26e9fcc32edd26747948e452d83e74739cae0b4709c2ff92bbac7e55023b621
Jun 30 01:38:21 proxmox2 proxmox-backup-proxy[14513]: upload_chunk done: 1879596 bytes, dd5dbb62a20c6d704491277022fc6f858f087fe38d0d9fdef6aa81e8be176927
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: backup failed: connection error: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: removing failed backup
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: TASK ERROR: connection error: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: removing backup snapshot "/mnt/datastore/Backup1/ns/PROXMOX3/ct/315/2024-06-29T23:00:00Z"
Jun 30 01:38:23 proxmox2 proxmox-backup-proxy[14513]: POST /dynamic_chunk: 400 Bad Request: error reading a body from connection: error:0A000119:SSL routines:ssl3_get_record:decryption failed or bad record mac:../ssl/record/ssl3_record.c:622:
Jun 30 01:47:12 proxmox2 proxmox-backup-proxy[14513]: write rrd data back to disk
Jun 30 01:47:12 proxmox2 proxmox-backup-proxy[14513]: starting rrd data sync
Jun 30 01:47:12 proxmox2 proxmox-backup-proxy[14513]: rrd journal successfully committed (25 files in 0.008 seconds)
Click to expand...
So there seems to be an issue while uploading the chunks to the Proxmox Backup Server. The error might indicate that there is something mangling with your upload http/2 requests. What VPN solution are you using for the connection, can you verify that the connection remains stable during the whole backup process?

Edit: Also, how fast is the connection to the remote? You could backup to the loacal PBS instance and create a sync job to pull the created snapshots to the remote host. This is normally the better approach to sync offsite backups, see https://pbs.proxmox.com/docs/managing-remotes.html#sync-jobs
 
Last edited:
Chris said:
... can you verify that the connection remains stable during the whole backup process?
Click to expand...
The connection uses IPSEC between a FritzBox and a PFsense Box. All I can see in the logs is, that the Fritzbox is heavily under load during the backup, but does not drop the connection at any point. Maybe there is a buffer that has an overflow, but the FB log is not that specific.
What protocol is used for the Backup-process, please? FTP/NFS?
 
Zaphood said:
The connection uses IPSEC between a FritzBox and a PFsense Box. All I can see in the logs is, that the Fritzbox is heavily under load during the backup, but does not drop the connection at any point. Maybe there is a buffer that has an overflow, but the FB log is not that specific.
What protocol is used for the Backup-process, please? FTP/NFS?
Click to expand...
You might be better off when using a sync job and/or setup a bandwidth limit to the PBS then. Maybe your Fritzbox cannot handle the IPSEC overhead? Nevertheless, you traffic somehow gets corrupted while being transferred to the remote PBS instance.

The backup protocol uses HTTP/2 for the upload of chunks.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back