buffer i/o error on dev zd96

[twentytwo]

This error appears after I restart the server and Windows 10 VM are not responsive, couple of programs that I use for work does not work. Could anyone help me diagonse this problem.

I have plenty of space on my hard disk so it has be something else that I am not aware of.

 

[fiona]

Hi,
could you share the exact error message and where you got the error? What does zpool status -v say? Is there anything related in the output of dmesg or in /var/log/syslog?

 

[twentytwo]

I cannot provide the log as I have start fresh installation and the vzdump backups saved in /dev/sdd. I tried to mount it and when I view the content which is empty.. How to I find out if I have any data on that drive?

 

[fiona]

After mounting the partition (with the instructions given by @t.lamprecht in the other thread), you may double check with findmnt /dev/sdd1 to see if it is really mounted. Then simply do a cd /mnt/bak and ls -a. If there is no output, then I'm sorry to say that there is no (accessible) data on the file system.

 

[twentytwo]

I found the backups files which is located in bak - backup - dump, whats the next step?

 

[fiona]

Good to hear!
You can add the directory /mnt/bak/backup as a storage to Proxmox. In the WebGUI go to Datacenter > Storage > Add > Directory choose a name, put in /mnt/bak/backup for Directory and set Content to VZDump backup file. Then the storage should appear in the sidebar and when you click on the storage and then on Content you should see the backup files and be able to restore them.

 

[twentytwo]

I try to restore first backup file and stop. Not sure what the exit code 133 mean?

estore vma archive: lzop -d -c /mnt/bak/backup/backup//dump/vzdump-qemu-100-2020_02_28-08_59_12.vma.lzo | vma extract -v -r /var/tmp/vzdumptmp15486.fifo - /var/tmp/vzdumptmp15486
CFG: size: 684 name: qemu-server.conf
DEV: dev_id=1 size: 1048576 devname: drive-efidisk0
DEV: dev_id=2 size: 214748364800 devname: drive-scsi0
CTIME: Fri Feb 28 08:59:15 2020
new volume ID is 'local-zfs:vm-100-disk-0'
map 'drive-efidisk0' to '/dev/zvol/rpool/data/vm-100-disk-0' (write zeros = 0)
new volume ID is 'local-zfs:vm-100-disk-1'
map 'drive-scsi0' to '/dev/zvol/rpool/data/vm-100-disk-1' (write zeros = 0)
progress 1% (read 2147549184 bytes, duration 7 sec)
progress 2% (read 4295032832 bytes, duration 16 sec)
progress 3% (read 6442516480 bytes, duration 24 sec)
progress 4% (read 8590000128 bytes, duration 32 sec)
progress 5% (read 10737483776 bytes, duration 40 sec)
progress 6% (read 12884967424 bytes, duration 53 sec)
progress 7% (read 15032516608 bytes, duration 63 sec)
progress 8% (read 17180000256 bytes, duration 75 sec)
progress 9% (read 19327483904 bytes, duration 89 sec)
progress 10% (read 21474967552 bytes, duration 103 sec)
progress 11% (read 23622451200 bytes, duration 118 sec)
progress 12% (read 25769934848 bytes, duration 133 sec)
progress 13% (read 27917484032 bytes, duration 148 sec)
progress 14% (read 30064967680 bytes, duration 162 sec)
progress 15% (read 32212451328 bytes, duration 179 sec)
progress 16% (read 34359934976 bytes, duration 194 sec)
progress 17% (read 36507418624 bytes, duration 208 sec)
progress 18% (read 38654902272 bytes, duration 222 sec)
progress 19% (read 40802451456 bytes, duration 239 sec)
progress 20% (read 42949935104 bytes, duration 254 sec)
progress 21% (read 45097418752 bytes, duration 268 sec)
progress 22% (read 47244902400 bytes, duration 280 sec)
progress 23% (read 49392386048 bytes, duration 293 sec)
progress 24% (read 51539869696 bytes, duration 308 sec)
progress 25% (read 53687353344 bytes, duration 326 sec)
progress 26% (read 55834902528 bytes, duration 341 sec)
progress 27% (read 57982386176 bytes, duration 358 sec)
lzop: /mnt/bak/backup/backup//dump/vzdump-qemu-100-2020_02_28-08_59_12.vma.lzo: Checksum error
vma: restore failed - short vma extent (3740160 < 3801600)
/bin/bash: line 1: 15489 Exit 1 lzop -d -c /mnt/bak/backup/backup//dump/vzdump-qemu-100-2020_02_28-08_59_12.vma.lzo
15490 Trace/breakpoint trap | vma extract -v -r /var/tmp/vzdumptmp15486.fifo - /var/tmp/vzdumptmp15486
temporary volume 'local-zfs:vm-100-disk-1' sucessfuly removed
temporary volume 'local-zfs:vm-100-disk-0' sucessfuly removed
no lock found trying to remove 'create' lock
TASK ERROR: command 'set -o pipefail && lzop -d -c /mnt/bak/backup/backup//dump/vzdump-qemu-100-2020_02_28-08_59_12.vma.lzo | vma extract -v -r /var/tmp/vzdumptmp15486.fifo - /var/tmp/vzdumptmp15486' failed: exit code 133

 

[fiona]

The relevant error is here:

lzop: /mnt/bak/backup/backup//dump/vzdump-qemu-100-2020_02_28-08_59_12.vma.lzo: Checksum error

This might mean that the file is corrupted. You can still try to force unpacking as suggested here by using

lzop -x --no-checksum BAKUP-NAME.vma.lzo

If this works try to restore from the .vma file instead of the .vma.lzo file, which should appear in the Web view after refreshing.

But please check your disks first. Do what I suggested in my first answer and also use smartctl -Ha /dev/XYZ, both for the drive with the backup files and the drives with the zpool.

 

[twentytwo]

Okay but I realise double forward dash and too many backup folder names due to my mistake. How can I remove BackUP from Storage and modify bak to Backup and change the path target from /mnt/bak to /mnt/backup/dump ? Will these delete my backups files? Sorry for asking dumb questions

 

[fiona]

Okay but I realise double forward dash and too many backup folder names due to my mistake. How can I remove BackUP from Storage and modify bak to Backup and change the path target from /mnt/bak to /mnt/backup/dump ? Will these delete my backups files? Sorry for asking dumb questions

If PVE is able to see the backup files, then I'd say that the storage is configured correctly. The restore process successfully starts, which means that things like the double slash are just cosmetic and not the real problem.

If you really want to change it, you can first remove the storage from PVE in Datacenter > Storage (this will not delete the files on the storage). Then unmount and re-mount the filesystem in the desired location and don't forget to change the entry in /etc/fstab to make the change persistent. Then move the dump directory to the desired location and re-add the storage with the correct path. E.g. if the backup files are in /mnt/backup/dump, use /mnt/backup for the path.

 

[twentytwo]

I am not sure if I follow, what I have so far is remove bak and backUP from Storage. Next step is to rm mnt/bak and mnt/backupz?. I could not use umount becuase these are not mount which I assume I unmounted previously via Storage?

Do you mean unmount /dev/sdd1 and mount it again and what should I write in /etc/fstab ?

 

[fiona]

I am not sure if I follow, what I have so far is remove bak and backUP from Storage. Next step is to rm mnt/bak and mnt/backupz?.

If those directories are empty, you can remove them.

I could not use umount becuase these are not mount which I assume I unmounted previously via Storage?

Do you mean unmount /dev/sdd1 and mount it again and what should I write in /etc/fstab ?

Yes, first do umount /dev/sdd1. Then you can remove the directory that was the previous mount point, i.e. /mnt/bak. But make sure it's empty beforehand. Then use the commands from here again, but with the name backup instead of bak. In your /etc/fstab simply update the path for the line

UUID=d8871cd7-11b1-4f75-8cb6-254a6120 /mnt/bak    ext4    defaults 0   2

Then the new mount point should be /mnt/backup and the directory for the backups will be /mnt/backup/dump.