Backup error - Input/output error

romuloalves

New Member
Nov 19, 2024
3
0
1
Hello,

I’m experiencing backup errors on 2 VMs across different nodes. Initially, I suspected an issue with the disks on node 01, but the errors have now started occurring on node 02 as well. These issues affect both local backups and those performed on the PBS.

1732032002468.png
 
Hi,

please check the systemd journal around the time of the backup for errors. Anything of interest logged there? Also, do check the smart values of you disks. Further, what filesystem is the VM disk located on? Is this a local storage or a shared storage?
 
Hi,

please check the systemd journal around the time of the backup for errors. Anything of interest logged there? Also, do check the smart values of you disks. Further, what filesystem is the VM disk located on? Is this a local storage or a shared storage?
Hi, would these be the logs? The storage is local.

Nov 19 11:31:32 proxmox-server pvedaemon[3984083]: INFO: starting new backup job: vzdump 102 --node proxmox-server --notification-mode auto --notes-template '{{guestname}}' --storage pbs01 --remove 0 --mode snapshot
Nov 19 11:31:32 proxmox-server pvedaemon[3984083]: INFO: Starting Backup of VM 102 (qemu)
Nov 19 11:32:20 proxmox-server systemd[1]: Starting apt-daily.service - Daily apt download activities...
Nov 19 11:32:20 proxmox-server systemd[1]: apt-daily.service: Deactivated successfully.
Nov 19 11:32:20 proxmox-server systemd[1]: Finished apt-daily.service - Daily apt download activities.
Nov 19 11:34:19 proxmox-server postfix/qmgr[3484355]: 43F352365E55: from=<root@proxmox-server.local>, size=11776, nrcpt=1 (queue active)
Nov 19 11:34:19 proxmox-server postfix/qmgr[3484355]: B14B922D32E2: from=<root@proxmox-server.local>, size=38158, nrcpt=1 (queue active)
Nov 19 11:34:19 proxmox-server postfix/error[3985605]: 43F352365E55: to=<romulo.alves@exemplo.com>, relay=none, delay=460, delays=460/0.01/0/0.01, dsn=4.4.1, status=deferred (delivery temporarily suspended: connect to exemplo-com-.mail.protection.outlook.com[52.101.10.16]:25: Connection timed out)
Nov 19 11:34:19 proxmox-server postfix/error[3985606]: B14B922D32E2: to=<romulo.alves@
<root@proxmox-server.local>, size=38158, nrcpt=1 (queue active)
Nov 19 11:34:19 proxmox-server postfix/error[3985605]: 43F352365E55: to=<romulo.alves@exemplo.com>, relay=none, delay=295982, delays=295982/0.01/0/0, dsn=4.4.1, status=deferred (delivery temporarily suspended: connect to exemplo-com.mail.protection.outlook.com[52.101.10.16]:25: Connection timed out)
Nov 19 11:34:38 proxmox-server pmxcfs[3780009]: [dcdb] notice: data verification successful
Nov 19 11:35:04 proxmox-server pmxcfs[3780009]: [status] notice: received log
Nov 19 11:35:54 proxmox-server kernel: BTRFS warning (device sda3): csum failed root 5 ino 24579934 off 87854161920 csum 0x76a60e03 expected csum 0xd0336f2c mirror 1
Nov 19 11:35:54 proxmox-server kernel: BTRFS error (device sda3): bdev /dev/sda3 errs: wr 0, rd 0, flush 0, corrupt 3440, gen 0
Nov 19 11:35:54 proxmox-server kernel: BTRFS warning (device sda3): direct IO failed ino 24579934 op 0x0 offset 0x1474842000 len 4096 err no 10
Nov 19 11:35:54 proxmox-server kernel: BTRFS warning (device sda3): csum failed root 5 ino 24579934 off 87854145536 csum 0x63515fd4 expected csum 0x686409d3 mirror 1
Nov 19 11:35:54 proxmox-server kernel: BTRFS error (device sda3): bdev /dev/sda3 errs: wr 0, rd 0, flush 0, corrupt 3441, gen 0
Nov 19 11:35:54 proxmox-server kernel: BTRFS warning (device sda3): direct IO failed ino 24579934 op 0x0 offset 0x147483e000 len 4096 err no 10
Nov 19 11:35:55 proxmox-server pvedaemon[3984083]: ERROR: Backup of VM 102 failed - job failed with err -5 - Input/output error
Nov 19 11:35:55 proxmox-server pvedaemon[3984083]: INFO: Backup job finished with errors
Nov 19 11:35:55 proxmox-server perl[3984083]: notified via target `mail-to-root`
Nov 19 11:35:55 proxmox-server pvedaemon[3984083]: job errors
 
Nov 19 11:35:54 proxmox-server kernel: BTRFS warning (device sda3): csum failed root 5 ino 24579934 off 87854161920 csum 0x76a60e03 expected csum 0xd0336f2c mirror 1
Nov 19 11:35:54 proxmox-server kernel: BTRFS error (device sda3): bdev /dev/sda3 errs: wr 0, rd 0, flush 0, corrupt 3440, gen 0
Nov 19 11:35:54 proxmox-server kernel: BTRFS warning (device sda3): direct IO failed ino 24579934 op 0x0 offset 0x1474842000 len 4096 err no 10
Nov 19 11:35:54 proxmox-server kernel: BTRFS warning (device sda3): csum failed root 5 ino 24579934 off 87854145536 csum 0x63515fd4 expected csum 0x686409d3 mirror 1
Nov 19 11:35:54 proxmox-server kernel: BTRFS error (device sda3): bdev /dev/sda3 errs: wr 0, rd 0, flush 0, corrupt 3441, gen 0
Nov 19 11:35:54 proxmox-server kernel: BTRFS warning (device sda3): direct IO failed ino 24579934 op 0x0 offset 0x147483e000 len 4096 err no 10
Nov 19 11:35:55 proxmox-server pvedaemon[3984083]: ERROR: Backup of VM 102 failed - job failed with err -5 - Input/output error
Well, it seems you have filesystem errors. Do you have any redundancy on this storage? What kernel is in use?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!