Issue Backing up PVE host with PBS

nacnud

Active Member
Dec 28, 2019
3
0
41
32
Hello, I recently installed PBS and got to work backing up two VMs. One went off without a hitch and the other I received the following error every time I backed it up (around 4% complete):

ERROR: job failed with err -5 - Input/output error

Upon looking into that error I stumbled upon the backup logs (/var/log/proxmox-backup/tasks/) on PBS and found the following issues relating to VM 100 (the VM with the backup issue):

Code:
2024-11-24T21:19:57-05:00: starting new backup on datastore 'backups' from ::ffff:192.168.1.201: "vm/100/2024-11-25T02:19:57Z"
2024-11-24T21:19:57-05:00: GET /previous: 400 Bad Request: no valid previous backup
2024-11-24T21:19:57-05:00: created new fixed index 1 ("vm/100/2024-11-25T02:19:57Z/drive-scsi0.img.fidx")
2024-11-24T21:19:57-05:00: add blob "/mnt/datastore/beck/vm/100/2024-11-25T02:19:57Z/qemu-server.conf.blob" (363 bytes, comp: 363)
2024-11-24T22:21:57-05:00: backup ended and finish failed: backup ended but finished flag is not set.
2024-11-24T22:21:57-05:00: removing unfinished backup
2024-11-24T22:21:57-05:00: TASK ERROR: backup ended but finished flag is not set.

2024-11-24T22:44:05-05:00: starting new backup on datastore 'backups' from ::ffff:192.168.1.201: "vm/100/2024-11-25T03:43:38Z"
2024-11-24T22:44:05-05:00: GET /previous: 400 Bad Request: no valid previous backup
2024-11-24T22:44:05-05:00: created new fixed index 1 ("vm/100/2024-11-25T03:43:38Z/drive-scsi0.img.fidx")
2024-11-24T22:44:05-05:00: add blob "/mnt/datastore/beck/vm/100/2024-11-25T03:43:38Z/qemu-server.conf.blob" (363 bytes, comp: 363)
2024-11-24T23:37:50-05:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: connection reset
2024-11-24T23:37:50-05:00: backup failed: connection error: connection reset
2024-11-24T23:37:50-05:00: removing failed backup
2024-11-24T23:37:50-05:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: connection reset
2024-11-24T23:37:50-05:00: TASK ERROR: connection error: connection reset

2024-11-25T05:25:36-05:00: starting new backup on datastore 'backups' from ::ffff:192.168.1.201: "vm/100/2024-11-25T10:25:36Z"
2024-11-25T05:25:36-05:00: GET /previous: 400 Bad Request: no valid previous backup
2024-11-25T05:25:36-05:00: created new fixed index 1 ("vm/100/2024-11-25T10:25:36Z/drive-scsi0.img.fidx")
2024-11-25T05:25:36-05:00: add blob "/mnt/datastore/beck/vm/100/2024-11-25T10:25:36Z/qemu-server.conf.blob" (363 bytes, comp: 363)
2024-11-25T06:19:26-05:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: connection reset
2024-11-25T06:19:26-05:00: backup failed: connection error: connection reset
2024-11-25T06:19:26-05:00: removing failed backup
2024-11-25T06:19:26-05:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: connection reset
2024-11-25T06:19:26-05:00: TASK ERROR: connection error: connection reset

I read online that there might be some files in the VM that are corrupt and deleting them may resolve it. I am also wondering if a port needs to be opened as VM100 has an active firewall whereas VM105 (successful backup) does not. I also have 1 damaged disk in my ZFS pool which needs to be replaced...not sure if that could have something to do with it as that's where the VMs reside.

Anybody have any ideas or further troubleshooting tips? TIA
 
Hello, I recently installed PBS and got to work backing up two VMs. One went off without a hitch and the other I received the following error every time I backed it up (around 4% complete):

ERROR: job failed with err -5 - Input/output error

Upon looking into that error I stumbled upon the backup logs (/var/log/proxmox-backup/tasks/) on PBS and found the following issues relating to VM 100 (the VM with the backup issue):

Code:
2024-11-24T21:19:57-05:00: starting new backup on datastore 'backups' from ::ffff:192.168.1.201: "vm/100/2024-11-25T02:19:57Z"
2024-11-24T21:19:57-05:00: GET /previous: 400 Bad Request: no valid previous backup
2024-11-24T21:19:57-05:00: created new fixed index 1 ("vm/100/2024-11-25T02:19:57Z/drive-scsi0.img.fidx")
2024-11-24T21:19:57-05:00: add blob "/mnt/datastore/beck/vm/100/2024-11-25T02:19:57Z/qemu-server.conf.blob" (363 bytes, comp: 363)
2024-11-24T22:21:57-05:00: backup ended and finish failed: backup ended but finished flag is not set.
2024-11-24T22:21:57-05:00: removing unfinished backup
2024-11-24T22:21:57-05:00: TASK ERROR: backup ended but finished flag is not set.

2024-11-24T22:44:05-05:00: starting new backup on datastore 'backups' from ::ffff:192.168.1.201: "vm/100/2024-11-25T03:43:38Z"
2024-11-24T22:44:05-05:00: GET /previous: 400 Bad Request: no valid previous backup
2024-11-24T22:44:05-05:00: created new fixed index 1 ("vm/100/2024-11-25T03:43:38Z/drive-scsi0.img.fidx")
2024-11-24T22:44:05-05:00: add blob "/mnt/datastore/beck/vm/100/2024-11-25T03:43:38Z/qemu-server.conf.blob" (363 bytes, comp: 363)
2024-11-24T23:37:50-05:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: connection reset
2024-11-24T23:37:50-05:00: backup failed: connection error: connection reset
2024-11-24T23:37:50-05:00: removing failed backup
2024-11-24T23:37:50-05:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: connection reset
2024-11-24T23:37:50-05:00: TASK ERROR: connection error: connection reset

2024-11-25T05:25:36-05:00: starting new backup on datastore 'backups' from ::ffff:192.168.1.201: "vm/100/2024-11-25T10:25:36Z"
2024-11-25T05:25:36-05:00: GET /previous: 400 Bad Request: no valid previous backup
2024-11-25T05:25:36-05:00: created new fixed index 1 ("vm/100/2024-11-25T10:25:36Z/drive-scsi0.img.fidx")
2024-11-25T05:25:36-05:00: add blob "/mnt/datastore/beck/vm/100/2024-11-25T10:25:36Z/qemu-server.conf.blob" (363 bytes, comp: 363)
2024-11-25T06:19:26-05:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: connection reset
2024-11-25T06:19:26-05:00: backup failed: connection error: connection reset
2024-11-25T06:19:26-05:00: removing failed backup
2024-11-25T06:19:26-05:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: connection reset
2024-11-25T06:19:26-05:00: TASK ERROR: connection error: connection reset

I read online that there might be some files in the VM that are corrupt and deleting them may resolve it. I am also wondering if a port needs to be opened as VM100 has an active firewall whereas VM105 (successful backup) does not. I also have 1 damaged disk in my ZFS pool which needs to be replaced...not sure if that could have something to do with it as that's where the VMs reside.

Anybody have any ideas or further troubleshooting tips? TIA
Hi,
what is your proxmox-backup-manager version --verbose? The IO error you get during backup of the VM could indicate a storage issue on the PVE side. Please check the systemd journal on the PVE for errors around the time of the backup and perform a disk smart check.
 
Hi,
what is your proxmox-backup-manager version --verbose? The IO error you get during backup of the VM could indicate a storage issue on the PVE side. Please check the systemd journal on the PVE for errors around the time of the backup and perform a disk smart check.
I do think it is on the PVE side because it backs up other VMs. My output is as follows:

xyxz@pbs:~# proxmox-backup-manager version --verbose
proxmox-backup 3.0.1 running kernel: 6.8.4-2-pve
proxmox-backup-server 3.2.2-1 running version: 3.2.2
proxmox-kernel-helper 8.1.0
proxmox-kernel-6.8 6.8.4-2
proxmox-kernel-6.8.4-2-pve-signed 6.8.4-2
ifupdown2 3.2.0-1+pmx8
libjs-extjs 7.0.0-4
proxmox-backup-docs 3.2.2-1
proxmox-backup-client 3.2.2-1
proxmox-mail-forward 0.2.3
proxmox-mini-journalreader 1.4.0
proxmox-offline-mirror-helper 0.6.6
proxmox-widget-toolkit 4.2.3
pve-xtermjs 5.3.0-3
smartmontools 7.3-pve1
zfsutils-linux 2.2.3-pve2

I will check your suggestions thank you.
 
Last edited: