Hi,
I have a proxmox server running 2 VM's: The first is 700Gb and the second is 32Gb.
I'm having the below errors (seems a bad sector) when trying to backup the first 700Gb machine to an external HDD while the second succeeded.
Any help is really appreciated as it is critical to have a backup of this machine.
here is the output of the task
Here is the error logs form journalctl
Here is the output of the fdisk -l
Also this :
Again thanks for any help in advance.
I have a proxmox server running 2 VM's: The first is 700Gb and the second is 32Gb.
I'm having the below errors (seems a bad sector) when trying to backup the first 700Gb machine to an external HDD while the second succeeded.
Any help is really appreciated as it is critical to have a backup of this machine.
here is the output of the task
Code:
INFO: starting new backup job: vzdump 100 --mode snapshot --remove 0 --node proxmox --storage local-usb --compress zstd
INFO: Starting Backup of VM 100 (qemu)
INFO: Backup started at 2023-04-08 12:34:14
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: UBUNTU-ELK
INFO: include disk 'scsi0' 'local-lvm:vm-100-disk-0' 700G
INFO: snapshots found (not included into backup)
INFO: creating vzdump archive '/mnt/usb-backup/dump/vzdump-qemu-100-2023_04_08-12_34_14.vma.zst'
INFO: starting kvm to execute backup task
INFO: started backup task '343066d5-99c1-4a59-ad63-2d5acdef144c'
INFO: 0% (446.0 MiB of 700.0 GiB) in 3s, read: 148.7 MiB/s, write: 103.0 MiB/s
INFO: 1% (7.1 GiB of 700.0 GiB) in 33s, read: 227.3 MiB/s, write: 84.6 MiB/s
INFO: 2% (14.1 GiB of 700.0 GiB) in 1m 44s, read: 101.1 MiB/s, write: 99.4 MiB/s
INFO: 3% (21.1 GiB of 700.0 GiB) in 2m 58s, read: 96.9 MiB/s, write: 94.9 MiB/s
INFO: 4% (28.0 GiB of 700.0 GiB) in 4m 10s, read: 98.6 MiB/s, write: 97.0 MiB/s
INFO: 5% (35.1 GiB of 700.0 GiB) in 5m 18s, read: 105.9 MiB/s, write: 103.9 MiB/s
INFO: 6% (42.0 GiB of 700.0 GiB) in 6m 27s, read: 103.1 MiB/s, write: 101.5 MiB/s
INFO: 7% (49.0 GiB of 700.0 GiB) in 7m 35s, read: 105.2 MiB/s, write: 103.6 MiB/s
INFO: 8% (56.0 GiB of 700.0 GiB) in 8m 42s, read: 107.1 MiB/s, write: 105.1 MiB/s
INFO: 9% (63.0 GiB of 700.0 GiB) in 9m 53s, read: 101.2 MiB/s, write: 99.3 MiB/s
INFO: 10% (70.1 GiB of 700.0 GiB) in 11m 5s, read: 100.4 MiB/s, write: 98.7 MiB/s
INFO: 11% (77.1 GiB of 700.0 GiB) in 12m 14s, read: 103.5 MiB/s, write: 101.5 MiB/s
INFO: 12% (84.1 GiB of 700.0 GiB) in 13m 25s, read: 101.3 MiB/s, write: 99.7 MiB/s
INFO: 13% (91.1 GiB of 700.0 GiB) in 14m 33s, read: 105.9 MiB/s, write: 103.5 MiB/s
INFO: 14% (98.0 GiB of 700.0 GiB) in 15m 43s, read: 100.9 MiB/s, write: 98.3 MiB/s
INFO: 14% (103.5 GiB of 700.0 GiB) in 16m 39s, read: 101.1 MiB/s, write: 98.6 MiB/s
ERROR: job failed with err -5 - Input/output error
INFO: aborting backup job
INFO: stopping kvm after backup task
trying to acquire lock...
OK
ERROR: Backup of VM 100 failed - job failed with err -5 - Input/output error
INFO: Failed at 2023-04-08 12:50:57
INFO: Backup job finished with errors
TASK ERROR: job errors
Here is the error logs form journalctl
Code:
Apr 5 15:07:00 proxmox systemd[1]: Started Proxmox VE replication runner.
Apr 5 15:07:16 proxmox kernel: [ 6471.305446] ata5.00: exception Emask 0x0 SAct 0x1000000 SErr 0x0 action 0x0
Apr 5 15:07:16 proxmox kernel: [ 6471.305449] ata5.00: irq_stat 0x40000008
Apr 5 15:07:16 proxmox kernel: [ 6471.305451] ata5.00: failed command: READ FPDMA QUEUED
Apr 5 15:07:16 proxmox kernel: [ 6471.305454] ata5.00: cmd 60/80:c0:00:c8:b2/00:00:35:00:00/40 tag 24 ncq dma 65536 in
Apr 5 15:07:16 proxmox kernel: [ 6471.305454] res 41/40:20:60:c8:b2/00:00:35:00:00/00 Emask 0x409 (media error) <F>
Apr 5 15:07:16 proxmox kernel: [ 6471.305456] ata5.00: status: { DRDY ERR }
Apr 5 15:07:16 proxmox kernel: [ 6471.305457] ata5.00: error: { UNC }
Apr 5 15:07:16 proxmox kernel: [ 6471.305792] ata5.00: configured for UDMA/133
Apr 5 15:07:16 proxmox kernel: [ 6471.305797] sd 4:0:0:0: [sda] tag#24 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Apr 5 15:07:16 proxmox kernel: [ 6471.305798] sd 4:0:0:0: [sda] tag#24 Sense Key : Medium Error [current]
Apr 5 15:07:16 proxmox kernel: [ 6471.305799] sd 4:0:0:0: [sda] tag#24 Add. Sense: Unrecovered read error - auto reallocate failed
Apr 5 15:07:16 proxmox kernel: [ 6471.305800] sd 4:0:0:0: [sda] tag#24 CDB: Read(10) 28 00 35 b2 c8 00 00 00 80 00
Apr 5 15:07:16 proxmox kernel: [ 6471.305802] blk_update_request: I/O error, dev sda, sector 900909152 op 0x0:(READ) flags 0x0 phys_seg 4 prio class 2
Apr 5 15:07:16 proxmox kernel: [ 6471.305810] ata5: EH complete
Apr 5 15:07:16 proxmox kernel: [ 6471.636281] fwbr100i0: port 2(tap100i0) entered disabled state
Apr 5 15:07:16 proxmox kernel: [ 6471.655016] fwbr100i0: port 1(fwln100i0) entered disabled state
Apr 5 15:07:16 proxmox kernel: [ 6471.655046] vmbr0: port 2(fwpr100p0) entered disabled state
Apr 5 15:07:16 proxmox kernel: [ 6471.655205] device fwln100i0 left promiscuous mode
Apr 5 15:07:16 proxmox kernel: [ 6471.655206] fwbr100i0: port 1(fwln100i0) entered disabled state
Apr 5 15:07:16 proxmox kernel: [ 6471.669532] device fwpr100p0 left promiscuous mode
Apr 5 15:07:16 proxmox kernel: [ 6471.669534] vmbr0: port 2(fwpr100p0) entered disabled state
Apr 5 15:07:16 proxmox qmeventd[734]: read: Connection reset by peer
Apr 5 15:07:16 proxmox pvedaemon[16712]: VM 100 qmp command failed - VM 100 not running
Apr 5 15:07:16 proxmox systemd[1]: 100.scope: Succeeded.
Apr 5 15:07:16 proxmox qmeventd[734]: Starting cleanup for 100
Apr 5 15:07:16 proxmox qmeventd[734]: trying to acquire lock...
Apr 5 15:07:17 proxmox qmeventd[734]: OK
Apr 5 15:07:17 proxmox qmeventd[734]: Finished cleanup for 100
Apr 5 15:07:21 proxmox pvedaemon[15351]: ERROR: Backup of VM 100 failed - job failed with err -5 - Input/output error
Apr 5 15:07:21 proxmox pvedaemon[15351]: INFO: Backup job finished with errors
Apr 5 15:07:21 proxmox pvedaemon[15351]: job errors
Apr 5 15:07:34 proxmox pveproxy[15201]: worker exit
Apr 5 15:07:34 proxmox pveproxy[1097]: worker 15201 finished
Apr 5 15:07:34 proxmox pveproxy[1097]: starting 1 worker(s)
Apr 5 15:07:34 proxmox pveproxy[1097]: worker 18022 started
Here is the output of the fdisk -l
Code:
root@proxmox:~# fdisk -l
Disk /dev/sda: 894.3 GiB, 960197124096 bytes, 1875385008 sectors
Disk model: Patriot Burst
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: E2DE254E-B9D9-479C-A9E6-33458B01D15D
Device Start End Sectors Size Type
/dev/sda1 34 2047 2014 1007K BIOS boot
/dev/sda2 2048 1050623 1048576 512M EFI System
/dev/sda3 1050624 1875384974 1874334351 893.8G Linux LVM
Disk /dev/sdb: 931.5 GiB, 1000204885504 bytes, 1953525167 sectors
Disk model: Expansion
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x800f1985
Device Boot Start End Sectors Size Id Type
/dev/sdb1 2048 1953521663 1953519616 931.5G 7 HPFS/NTFS/exFAT
Disk /dev/mapper/pve-swap: 8 GiB, 8589934592 bytes, 16777216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/pve-root: 96 GiB, 103079215104 bytes, 201326592 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/pve-vm--100--disk--0: 700 GiB, 751619276800 bytes, 1468006400 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes
Disklabel type: gpt
Disk identifier: 9E29377F-B8C3-468B-9972-CA36DF58A3A2
Device Start End Sectors Size Type
/dev/mapper/pve-vm--100--disk--0-part1 2048 4095 2048 1M BIOS boot
/dev/mapper/pve-vm--100--disk--0-part2 4096 2101247 2097152 1G Linux file
/dev/mapper/pve-vm--100--disk--0-part3 2101248 1468004351 1465903104 699G Linux file
Disk /dev/mapper/pve-vm--100--state--BeforeUpdate: 51.4 GiB, 55150903296 bytes, 107716608 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes
Disk /dev/mapper/pve-vm--101--disk--0: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes
Disklabel type: gpt
Disk identifier: 05DCF76C-5996-473A-81AD-8EB32A4D6C19
Device Start End Sectors Size Type
/dev/mapper/pve-vm--101--disk--0-part1 2048 4095 2048 1M BIOS boot
/dev/mapper/pve-vm--101--disk--0-part2 4096 2101247 2097152 1G Linux filesyste
/dev/mapper/pve-vm--101--disk--0-part3 2101248 67106815 65005568 31G Linux filesyste
Disk /dev/mapper/pve-vm--101--state--FleetSRVSuccess: 6.4 GiB, 6832521216 bytes, 13344768 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes
Disk /dev/mapper/pve-vm--100--state--FleetSucess: 51.4 GiB, 55150903296 bytes, 107716608 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes
Also this :
Code:
qemu-img: error while reading at byte 111176292864: Input/output error
Again thanks for any help in advance.