Backup error - job failed with err -5 - Input/output error

[resimarc]

Hello everyone.

We seem to be having storage issues with our Production servers. ZFS is telling us a disk has failed (the server RAID card says the disk is OK!), and it insists that we need to create the ZFS storage again.

We are trying to backup all data, but it is impossible, it gives an error "job failed with err -5 - Input/output error"

So basically, we cannot backup any info because it refuses to do so, and its just telling us to start over which is impossible

Proxmox says we should restore the entire pool from backup (impossible as we cannot backup)

Some services are affected as they give random I/O errors too

Any idea of what we can do besides

I attach backup logs

402: 2025-04-05 11:19:28 INFO: 27% (2.0 TiB of 7.5 TiB) in 2h 2m 17s, read: 265.4 MiB/s, write: 175.0 MiB/s
402: 2025-04-05 11:24:10 INFO: 28% (2.1 TiB of 7.5 TiB) in 2h 6m 59s, read: 281.0 MiB/s, write: 175.3 MiB/s
402: 2025-04-05 11:27:57 INFO: 28% (2.2 TiB of 7.5 TiB) in 2h 10m 46s, read: 326.8 MiB/s, write: 153.7 MiB/s
402: 2025-04-05 11:27:57 ERROR: job failed with err -5 - Input/output error
402: 2025-04-05 11:27:57 INFO: aborting backup job
402: 2025-04-05 11:27:57 INFO: resuming VM again
402: 2025-04-05 11:27:57 ERROR: Backup of VM 402 failed - job failed with err -5 - Input/output error

 

[fiona]

Hi,
if your disk is broken and cannot be fully read, you'll need special tools to salvage the remaining good data. E.g. have a look at man qemu-img/qemu-img convert --salvage or https://www.gnu.org/software/ddrescue/